I’m about halfway through the first day of Unified Computing System (UCS) training in San Jose, CA, and I’ve learned of what I think is a fairly significant limitation. The issue centers around what Cisco refers to as “northbound” traffic and how Fibre Channel over Ethernet (FCoE) is handled with northbound traffic.
Recall that a central part of UCS is the UCS 6100 series fabric interconnect. The 6100 series fabric interconnect has connectivity in two directions:
- Southbound connectivity is connectivity aimed back at the fabric extenders in the blade chassis themselves.
- Northbound connectivity is connectivity headed outside the UCS to other systems and networks.
All southbound traffic is 10Gbps Ethernet with FCoE. Northbound traffic can be 10Gbps Ethernet or Fibre Channel, but not FCoE. Based on the information I’ve been given (and if I’m incorrect please let me know in the comments), you cannot directly connect an FCoE-enabled storage array to a UCS. Even if your storage array has native FCoE interfaces, you can’t plug them into the UCS 6100 series fabric interconnects because that’s considered northbound traffic and you can’t use FCoE with northbound traffic.
I have a feeling customers who have purchased storage arrays with FCoE interfaces with the intention of hooking the arrays up directly to a UCS are going to be a bit upset when this information becomes more widely known.
If I’m working from incorrect or incomplete information, please feel free to speak up in the comments.
Tags: Cisco, FCoE, FibreChannel, Hardware, Networking, UCS
-
Ask if you can plug your array right into the 6100 to have support. That seems like a reasonable compromise. (I meant to ask this question when I was in training)
-
Russell,
The current UCS Manager software operates the FC functions in NPV mode, not FC switch mode. Therefore all zoning and FC forwarding decisions are made by the upstream SAN switch. When UCS software supports FC switch mode you will be able to connect an FC storage array directly into the UCS Manager.Cheers,
Brad -
Thanks for the clarification Brad! I actually remember something along those lines when I was in the training but I had forgotten. I should refer to my notes more frequently.
Regarding the whole “you gotta have a nexus!” thing; Cisco made it very clear in my UCS training that they don’t care if your network upstream is Brocade/Foundary or even HP Procurve with a QLogic SAN. The 6100 hands off standard 10Gbe and standard FCP. I think the idea is that a lot of people will be plugging these in to Cat 6500s and MDS switches.
-
I don’t find this a limitation at all, but an interesting feature. Basically, what you are asking for is that UCS becomes a FC switch. And it is just not that (currently). It is not a fabric switch, it just represents multiple hosts on an N port (using NPIV, i am no FC expert at all…). And honestly, would you really want a Cisco fabric switch in your SAN if you are standarized on Brocade or something else ? i guess not, so it keeps the flexibility to connect ucs to no matter what. The same goes for the LAN side, besides. You don’t need a nexus at all, this comment is a little “biased”
-
Could you attach the UCS to a N5k and attach the FC storage to the 5k and run FCoE?
-
Hello,
i have to confirm Scotts worries. I assume with new version that they will allow 10G FCoE all across datacenter. This is abstract from UCS Config guide 1.3
“Network Type
The network type is only relevant to traffic on uplink ports, because FCoE does not exist outside Cisco UCS. The rest of the data center network only differentiates between LAN and SAN traffic. Therefore, you do not need to take the network type into consideration when you estimate oversubscription of a fabric interconnect port.”and
“Cisco UCS uses Fibre Channel over Ethernet (FCoE) to carry Fibre Channel and Ethernet traffic on the same physical Ethernet connection between the fabric interconnect and the server. This connection terminates at a
converged network adapter on the server, and the unified fabric terminates on the uplink ports of the fabric interconnect. On the core network, the LAN and SAN traffic remains separated. Cisco UCS does not require that you implement unified fabric across the data center.”With regards.
Bozo -
Has anyone heard any updates as to when Cisco may begin to offer Northbound FCOE functionality? Even better if it could be offered in Switch mode.
-
I’m finding this a pretty severe limitation as well. I’m not understanding the limitation of making an FCoE port as an NP port upstream to an NPIV FCoE switch like the Nexus 5k. Why does the uplink port have to be native FC? I don’t mind not having switching in the interconnects but it would save a lot of money if I could use FCoE uplinks as an NP port to the 5k and have it act as my FC switch and connect to my FC array(s). I have a lot of customers that are looking to replace their 5-6 year old FC switches and I could justify selling N5K’s as replacements.
Any chance at all this will be released in the update coming in December?
-
Pingback from Cisco Nexus 5000 Vs. UCS 6100 on Friday, November 19, 2010 at 8:27 pm
-
Jean, I do not think it northbound FCoE is here yet. I could not find any blog posts or discussion about it and I would think it would be a pretty big deal which would have seen some discussion. I looked over the public release notes for UCS manager on Cisco’s site all the way up to 2.0 and did not see anything, hoping I missed it somewhere but I am not holding my breath. Time to add a few FC cables to my diagram.
-
UCS does not support FCoE Northbound connectivity yet.
With a Service Profile, you can only assign regular NICs (10gb), and FC HBA’s, you can’t assign a CNA adapter to a service profile.




22 comments
Comments feed for this article
Trackback link: http://blog.scottlowe.org/2009/07/27/potential-ucs-issue-northbound-fcoe-connectivity/trackback/