I ran into a recent issue with a customer who was having problems getting VLANs to work as expected with ESX. The basic scenario was that ESX would refuse to work properly with a VLAN that was marked as the native (or untagged) VLAN. This was causing no end of grief for this customer.
I’ve discussed VLANs extensively—first with this blog post, then again here, and again in this SearchVMware.com tip—so I was confident that I could help the customer resolve this issue. Granted, the customer was using Nortel switches, with which I am completely unfamiliar, but a switch is a switch, right?
Not quite. While the configuration seemed correct in all ways, it turns out there is a checkbox somewhere labeled “untag-default-vlan”. If this box is not checked, then the default VLAN gets tagged. Since ESX wasn’t configured with a VLAN tag, then it doesn’t see the network traffic. Once that box gets checked, then the default (or native) VLAN doesn’t get tagged and will be properly recognized by an ESX port group without a VLAN tag configured.
So, if you’re using Nortel switches and having problems with VLANs, double-check this setting.
Tags: ESX, Networking, Virtualization, VLAN, VMware
-
If you’re using the CLI on a 55xx switch, you can ensure that the VLAN connecting to the VMware box is untagged by setting the port to “untagAll”. The command is:
vlan ports tagging untagAllIf you have multiple VLANs on the port and you want the default VLAN (called PVID) to be untagged you do:
vlan ports tagging untagPvidOnlyTo set the default VLAN on a port, you do:
vlan ports <port(s)> pvid <vlanID>Here’s a user guide describing in detail how to set up a resilient network configuration for VMware ESX using Nortel’s SMLT:
http://support.nortel.com/go/main.jsp?cscat=DOCDETAIL&id=678681&poid=14761 -
I ran into this exact same problem about a year ago. It drove me crazy, especially because I’m no networking guy… But in the end someone solved it.
-
Hello, anyone with Nortels I would love to know this. On our 3510 which is just like the 5510 but no stacking Intel cards always show asymmetrical flow control even though they can do symmetrical (send and receive pause frames)
Broadcoms show symmetrical
Anyone else see this at his/her site?
-
Hello Guys,
I’m currently, setting up an Vi3.5 with VC 2.5 and having problem in getting the VLAN to work with Nortel Switch 4542GT.My Setup:
2 DL380 each with 6 pNICs, connected to 2 Nortel Gig Switch in Stack Mode.Created 2 VLANs for the Service Console Portgroup and VMotion Portgroup.
vSwitch0 connected with vmnic0 & vmnic1
Created the VLAN 2 for the Service Console and connected vmnic0 to the port where i have created the VLAN on 172.16.20.0/24 segment.
Created VLAN 3 for the VMotion Portgroup and connected to the same vSiwtch0. 10.1.0.0/24
VLAN 2: 172.16.20.0/25 D.G: 172.16.20.1
VLAN 3: 10.1.0.0/24 D.G 10.1.0.1Both the Cables of the ESX Host are connected to each VLAN.
From the pSwitch, I’m able to ping both VLANs.
From the ESX Host, I can ping only other hosts on VLAN 2. But, I cannot ping the VMotion Portgroup on VLAN 3 using vmkping.
If, I put the VLAN Number into the VLAN ID in the Vi Client, I lose the connectivity to the ESX Host. and again, I have to assign another IP Address to different vmswif to get it work
and remove the VLAN ID from the vmswif0. My question is,, How i can get this to work. If vmnic0 goes down, I still can reach to the Service Console on VLAN 2 using VLAN 3?
Note: These two vLANs are different than the production VLAN where we are set. Also, I want to reach the Service Console VLAN via the production VLAN…
Any help?
Thanks,
-
Habibalby,
Let me make sure your configuration is correct–you should have two port groups on the same vSwitch. One port group, the Service Console port group, is tagged for VLAN 2 (or has no VLAN tag is VLAN 2 is set as the untagged/native VLAN). The other port group, the VMotion port group, is tagged for VLAN 3. The physical switch ports should be set as 802.1Q VLAN trunks.
Is all that correct?
-
Hello Slowe,
Thanks for your reply.
Yes, I do have a vSwitch0 which has two portgroups, for Service console and VMotion. All the ports in the switch are set to UnTagAll. If I set the VLAN id in Server Console or VMotion portgroup, I lose the connectivity to the ESX host.
If I leave the VLAN id in the Vi Client empty or set to 0, VMotion network cannot talk with Service Console, nor the Service Console sees the VMotion network.
Regarding the Trunking, I have to check with our Network Engineer to nake sure that the ports are set as 802.1Q trunks. Or if you can tell me how to set it up, I will be glad to…
-
Hello,
Here is in details my setup and what i want to achive with the Nortel Switches 4542GT in Cluster Stack-Mode.Server Configuration:
2- DL380 G5, each with Single Port HBA, 6 pNICs, 2 pCPU Dual-Proc.
2- BL460 G1, Each with Dual-port HBA, 6 pNICs, 1 pCPU Dual-Proc.Setup:
vSwitch0 = ESX Networks: Service Console “172.16.20.0/24″ && VMotion “10.1.0.0/24″ using VLANs.
vSwitch1= Production Network: 128.104.0.0/16
vSwitch2 = DMZ Network: 192.168.1.0/24Private Network for ESX:
vSwitch0 with 2 pNICs connected vmnic0 & vmnic1 Teamed on the vSwitch Level.
2 Portsgroup.
1 Service Console
1 VMotionIn the portgroup Setting for S.C –> Nic Teaming is vmnic0 Active and vmnic1 Standby
In the portgroup setting for VMotion –> Nic Teaming is the vmnic1 Active and vmnic0 Standby.vmnic0 connected to pSwitch on port configured with VLAN 2
vmnic1 connected to pSwitch on port configured with VLAN 3Production Network:
vSwitch1 with 2 pNICs connected vmnic2 & vmnic3 Teamed on the vSwitch Level.
1 Portgroup.
Production VMsvSwitch2 with 2 pNICs connected vmnic4 & vmnic5 Teamed on the vSwitch Level.
1 Portgroup
DMZ VMs
==============================================================================================================
If I assign an IP Address to the S.C with the same IP which is configured on the VLAN, “Without Assigning the ((VLAN ID)) in the portgourp, through pServer, i can reach to other ESX Host Service Console, because both of them are on the same VLAN.
As soon as I assign the ((VLAN ID)) on the portgroup of S.C, i lost the connectivity to the server, and I started troubleshooting the vswif0 to create another Service Console Network in order to access it the ESX Host. “And the same applies on the VMotion Network”.
The same goes to the VMotion network as well. From the pSwitch, both the VLANs are reachable to 172.16.20.0/24 Service Console, and 10.1.0.0/24 for VMotion Network.
I want the Service Console Network, can talk to the VMotion Network and vice versa to get the VMotion works.
Service Console:
IP:172.16.20.2/24
D.G: 172.16.20.1
DNS: 172.16.20.57 “This host is connected to the same VLAN where the ESX hosts connected”. It’s a VC and DNS Server.VMkarnal:
IP:10.1.0.2/24
D.G: 10.1.0.1From, within the ESX Host, I’m unable to reach to the Default Gateway of the VMotion Network using vmkping. Nor the Service Console able to reach to the VMotion Network.
Moreover, I wanted to reach to the Service Console Network 172.16.20.0 via 128.104.0.0 Network to do my Administrative Task. In this case, do I have to add a Static Route in the Service Console, in order for the VI Clients reach from Production Network?
Thank you very much.
Best Regards,
-
Hello,
More troubleshooting i have made;
I have UnTagged the ports for both VLANs. I setup both PortGroups S.C & VMKernel without VLAN ID.
I got one host can ping VMkernel PortGroup on another host via COS ping. Also from the same host tried vmkping S.C IP and D.Gateway. It’s successul.However, from the another host I can reach the first host S.C IP but not VMkernel. Nor the VMKernel able to reach it’s D.Gateway.
Since the both VLANs are reachable within the pSwitch. Do I have to use a port Trunking, and assign different VLAN ID “The Trunked vLAN” in each Portgroup *S.C & VMotion*?
In additional to what i have mentioned to earlier regarding the NIC Teaming. Both, vmnic0 & vmnic1 assigned to vSwitch0, in the NIC Teaming Setting of vSwitch0, both vmnic0 & vmnic1 as Active/Active. And within each PortGroup, S.C = vmnic0 Active & vmnic1 Standby. And VMkernel = vmnic1 Active & vmnic0 Standby. Is this Setting may confusing the VLAN to work properly?
Furter troubleshooting I’m going to make;
1. Remove the Nic Teaming from the PortGroups.
2. Configure the vSwitch0 with only vmnic0 on both hosts, assuming a pNIC failure.
3. Test both hosts can ping each other S.C and D.Gateway 172.16.20.1
4. Cofigure VMkernel with the prospetive VLAN IP schema & test vmkping whether it can reach S.C IP & it’s D.Gateway.
5. If it’s success, then will configure the same on the other host and test the connectivity between the hosts.If not, do I have to configure a Trunking on the pSwitches and make both VLANs 3 & 4 members of the Trunked VLAN?
Thank you all for your support.
Regards,
Site Sponsors
8 comments
Comments feed for this article
Trackback link: http://blog.scottlowe.org/2008/06/24/vlans-on-esx-with-nortel-switches/trackback/