Correction: There Might be an End-to-End FCoE Solution

This post is a follow-up from my post yesterday titled “No Such Thing as an End-to-End FCoE Solution”.

After publishing that post, I managed to get in touch with some very smart people who were willing to spend some time with me and educate me on the various intricacies involved here. In order to help you, my readers, understand the various pieces and parts, I’ll need to first provide some definitions.

Fibre Channel Forwarder (FCF): In its simplest form, this is another form for an FCoE switch. A Nexus 5000 would be an example of an FCF.

Multi-hop FCoE: There are a couple of different definitions here. One definition would be having multiple FCFs connected together (i.e., a Nexus 5000 connected to another Nexus 5000). A second definition would be having multiple Layer 2 hops between an FCoE initiator or target and an FCF. Note that the switches handling those hops must be IEEE DCB capable.

FCoE Initialization Protocol: FIP, as its more commonly known, was included in the FC-BB-5 FCoE standard that was finalized in early June.

OK, now that I have some definitions in place, I can discuss how it might be possible (eventually) to build an end-to-end FCoE solution.

Disclaimer: I don’t claim to be an FCoE expert, I’m just trying to understand it better myself and help others understand it better. If I’m misrepresenting something, let me know—courteously and professionally—in the comments, or drop me an e-mail.

The question “Can I build an end-to-end FCoE solution?” has multiple answers:

  • If you have only a single FCF and everything is plugged into that FCF, then you can build a pure FCoE solution today. The Nexus 5000 can function as the FCF, and both the CNAs and targets that are available will work. Obviously, this is not a very scalable solution.
  • If you have multiple FCFs, or if you have multiple Layer 2 hops between initiators or targets and the FCFs, then you might or might not be able to build an end-to-end FCoE solution. In this scenario, FIP-enabled initiators and targets would be able to find and communicate with each other, but non-FIP-enabled initiators and targets would not (unless they were plugged into the same FCF). At this point I am unclear about connectivity between pre-FIP initiators or targets on the same IEEE DCB-capable Layer 2 switch (not an FCF); I suspect they would not be able to communicate.

All of these statements are applicable until you bring UCS into the mix. For UCS, my earlier statement stands: with UCS, you cannot have an end-to-end FCoE solution today. That will change at some point in the future, but no one has shared any information with me regarding just how far in the future that might be.

If you have a pure Nexus 5000 environment, with FCoE-capable storage and servers with CNAs, you’d probably be able to make it work. With FIP support in that environment, you’d definitely be able to make it work. When you add UCS, though, it becomes very different. I hope to be able to discuss that in greater detail in the near future.

So, my earlier statement wasn’t entirely true; it is possible to build an end-to-end FCoE solution. Today, that solution would be very limited in size; once FIP support is baked into the initiators, the targets, and the FCFs, then the solution size will be able to scale.

As always, comments and clarifications are welcome!

Tags: , , , , ,

13 comments

  1. Justin’s avatar

    From my understanding, the UCS 6100 Fabric Interconnect is based on the Nexus 5000. So it makes me wonder why the UCS 6100 does not have all the functionality of the Nexus 5000 today.

  2. slowe’s avatar

    The UCS 6100 fabric interconnects are based on Nexus 5000 technologies, but the two products are not the same. Trust me, I’ve had the same discussions with the Cisco UCS folks. For now, you can’t make comparisons between the UCS 6100 fabric interconnects and the Nexus 5000 family of switches.

  3. Dejan Ilic’s avatar

    Hello,
    I was wondering what FIP was used for, as you text implies the possibility of existence of devices without it that can communicate, and found that there is a Wikipedia-page explaining some definition.

    http://en.wikipedia.org/wiki/Fibre_Channel_over_Ethernet

    You might want to include a link to it in your pages regarding FCoE.
    Regarding FIP, I’m still not sure what the purpose is (beside the obvious) and which protocol it replaces.

  4. Brad Hedlund’s avatar

    Scott,
    Excellent summary! You may not claim to be an FCoE expert but you are becoming one quickly.

    As for your statement: “…pre-FIP initiators or targets on the same IEEE DCB-capable Layer 2 switch (not an FCF); I suspect they would not be able to communicate.”

    You are correct!

    Unless, if the pre-FIP initiators and targets are connected to a “Fabric Extender”, then things start to get interesting :-)

    Cheers,
    Brad

  5. slowe’s avatar

    Yes, fabric extenders change the story quite dramatically. I’ll need a separate post for that!

  6. Serge Meeuwsen’s avatar

    Scott,
    Thanks again for keeping us all informed, very useful and educational indeed.

    With regards to UCS end-to-end (scalable) support for FCoE, I would assume that all that would required at a future date is updating firmware and such of all interconnecting devices? Though that might be a big assumption… :-)

    Cheers,
    Serge M

  7. Craig’s avatar

    As for today, UCS do not provide end to end FCoE due to the limitation apply on the UCS 6100. If you run the nexus 5000, CNA adapter and FCoE interface on your storage, you will have the end to end solution.

  8. slowe’s avatar

    Craig,

    You can run the Nexus 5000, CNAs in your servers, and FCoE interfaces on the storage and use end-to-end FCoE *AS LONG AS* you don’t exceed the switch port capacity of a Nexus 5000. If you need more ports than a single N5K can provide, you can’t add another N5K and continue to get end-to-end FCoE. Cisco is adding/has added FIP (FCoE Initialization Protocol) support to NX-OS in a new release for the N5K, but as far as I am aware the issue of transporting VLAN/VSAN IDs across VE_Ports between two N5Ks has not yet been addressed.

  9. jd’s avatar

    Scott,

    Great blog. Thanks for all the info.

    What’s your take on the current state of the standards at play here, FC-BB-5 and IEEE’s DCB (or CEE). DCB hasn’t been finalized. Should that concern anyone considering adopting this technology? Is the fact that FC-BB-5 is in draft format and is not ANSI standard at this point just a formality? Thanks.

  10. slowe’s avatar

    JD,

    The standardization of DCB/CEE will lead to multi-vendor FCoE interoperability. Even though FC-BB-5 has been finalized (not yet a formal standard, but finalized) and the “FC” side of FCoE has been locked down for multiple vendors to implement, the “oE” side of FCoE depends upon DCB/CEE. When that is finalized, I expect that more vendors will jump on board and uptake of the technology will increase significantly.

  11. Bud’s avatar

    Wow this post is over a year old and still no End-to-End connectivity from the UCS B-Series to storage. Cisco’s marketing hype on converged networks is confusing.

  12. Jackofalltrades’s avatar

    Hello,

    Is it possible to have a FCOE end to end solution based on mixing FCF like Nexus 5000 and Cisco MDS ( like FCOE ports from 5000 and FCOE ports from MDS interconnected to each other)?
    Please explain!!

Comments are now closed.