Comments on: NetApp VIF Member Limitations

By: Murthy

Murthy — Wed, 14 Sep 2011 01:44:12 +0000

Hi Jim,

The “flowcontrol send” has to be set on the individual interface like e0a etc. The vif will acquire this setting automatically from the interfaces.

Regds,
Murthy

By: Jim Bartlett

Jim Bartlett — Tue, 22 Feb 2011 22:00:52 +0000

I ran into an issue configuring flow control on my FAS3140 (7.3.3.P5). When attempting to set flow control on a vif, an error is spit out by ONTAP citing an “error 22, invalid argument to SIOCSFLOWCONTROL”. Looking up the KB at NOW there is a statement that says flow control is not supported on vifs, period.
But all the best practices guides say to set flow control on the filer (and in this case, the ESXi hosts) to send on, receive off. The Cisco switch should be send off, receive desired.

SO… if vifs do not support flow control, what is the real “best practice”?

By: Derric

Derric — Thu, 30 Jul 2009 18:06:21 +0000

Was this issue ever resolved? Im seeing the same behavior on a new FAS2050 install. Onboard nics work fine together in both multi and lacp vifs. Addon nics work fine together in both multi or lacp vifs. Put one of each in the vif, and both links refuse to come up.

By: slowe

slowe — Tue, 12 May 2009 19:39:08 +0000

I forwarded the case number over to a couple of contacts at NetApp but never heard anything back. If you want more information, e-mail me and I’ll see what I can do.

By: Mike

Mike — Tue, 12 May 2009 18:42:07 +0000

SLOWE – what was your case number?

By: slowe

slowe — Fri, 27 Feb 2009 04:11:23 +0000

Peter, thanks for the response and the additional information. I’m working to track down the specific case number so I can get it back to Vaughn, Eric, and others; then we can really try to nail down exactly what’s happening.

By: Peter Learmonth

Peter Learmonth — Fri, 27 Feb 2009 03:50:00 +0000

Hi gang
I’m one of the NetApp peeps talking to Scott in the background. I’m politely calling “negatory” on this in general. Mostly. However, I looked up 295540 and the issue is a hard-coded-on-the-card negotiation issue on certain cards where the cards start their negotiation at 10Mbps. When the other ports start at 1Gbps, the switch disables the 10Mbps ports. There are workarounds in the public burt. Not sure why the burt isn’t published, but I’ve asked. One workaround is “When including the listed controllers in a mixed-controller VIF, set the
speed on all switch ports in the VIF. The link will come up even though the controllers only support autonegotiation at 1000 Mbps.”

I’ve seen best practices, including from VMware, that recommend not mixing cards from different vendors (or even different family or model from the same vendor) within an active-active team/channel/bond. While not mixing cards is one workaround of 295540, there is no hard rule not to mix on-board and PCI slot interfaces in a multi-mode or LACP VIF. Oh, and it’s not about whether you mix on-board and slots, it’s about the Ethernet chips in question, so look at card description in sysconfig. Couple examples:

FAS3070 with an additional dual-port
slot 0: BGE 10/100/1000 Ethernet Controller
slot 2: Dual 10/100/1000 Ethernet Controller G20

FAS3020 with an additional dual-port
slot 0: Dual 10/100/1000 Ethernet Controller VI
slot 1: Dual Gigabit Ethernet Controller VI

The FAS3070 could have this issue (but I’m not). The FAS3020 has the same chips on-board and in slot, so should never have this problem. (It says dual for the motherboard when it’s actually 4 ports because it’s 2 dual-port chips.)

Another way to fix this problem would be for Cisco not to disable the port right away. Give the NIC a chance to negotiate up to a better speed.

I’m currently working on a RefArch with much the same scenario everyone else seems to be piling on:
Multiple ESX 3.5u2/u3 Pair of stacked c3750 FAS3070HA

On the filer side, it’s two onboard ports and both ports of a dual port (happens to be a G20 supposedly affected by 295540, but I haven’t seen that) in an LACP VIF.

Here’s the pertinent part of /etc/rc:

vif create lacp vif-stor -b ip e0b e0d e2a e2b
ifconfig e0a 10.60.120.189 mediatype auto netmask 255.255.255.0 partner e0a
ifconfig vif-stor 192.168.42.203 mediatype auto netmask 255.255.255.0 partner vif-stor
ifconfig vif-stor alias 192.168.42.204 netmask 255.255.255.0

We’ve tested this by unplugging cables (ESX side and filer side), 1 and 2 at a time, unplugging a switch, and powering off a head and it just doesn’t seem to break.

Share and enjoy!

Peter

By: Dan C

Dan C — Mon, 09 Feb 2009 23:52:50 +0000

Haven’t been able to replicate what I saw with some quick tests just now. Although we’re on a different ONTAP version since I last experienced it.

Hypothetically, if it was a problem with mixed PIFs, then some up-delay logic would prevent it. Similar to the behaviour of Linux’s bonding module which alleviates slaves from becoming active before a switch port moves into STP forwarding.

Also, from memory, ONTAP doesn’t allow you to hardset ports at 1GbE. If the switch port is set alone then are you going to experience issues with duplex and flow control negotiation?

By: slowe

slowe — Mon, 09 Feb 2009 23:19:08 +0000

Note to everyone: I’m getting a lot of pushback from NetApp folks saying this just isn’t right. I honestly don’t know the truth of the matter, but I’m working closely with several contacts within NetApp to try to determine exactly what’s going on. Thanks!

By: Dan C

Dan C — Mon, 09 Feb 2009 22:45:03 +0000

An interesting thread of comments. Thanks for sharing.

Just to add to the pot; I have before experienced an LACP VIF that wouldn’t return from a CF Giveback. Just the one inparticular which happens to span onboard/PCI PIFs and connected to 3com switches. Interface status looked OK but was unreachable by client hosts. An ifconfig up/down resolved it at the time.

One to watch I guess.