Cisco’s Data Center Launch

Cisco had a pretty major product/technology launch today. I was briefed on the product launch some time ago—one of the perks of being a “leading blogger”—and so it was great to see this stuff finally come to light. I haven’t had the chance to read some of the other blog posts on the topic (I know there have been several), so I apologize for any duplication.

First, let’s take a quick look at the “new product” news included in the launch:

  • Cisco is launching the Nexus 3000, a new member of the Nexus family. It’s a high density, ultra-low latency 1RU switch that supports both 1GbE as well as 10GbE. All ports are wire rate and non-blocking. At first glance, this seemed to be a bit of a conflict with some of the Catalyst switches (like the 4948, perhaps?), but I think the distinction here is the low latency target (approximately 1 usec port to port). It also makes sense if customers are interested in standardizing on switches running NX-OS instead of a mix of NX-OS and IOS.
  • Cisco is also launching new Nexus 5500 series switches, the Nexus 5548UP and Nexus 5596UP. These are very much like the other members of the Nexus 5500 family, except that the “UP” designation indicates that they will support “Unified Ports”. Unified Ports are ports that can support either Fibre Channel (supporting up to 8Gbps FC), Fibre Channel over Ethernet (at 10Gbps), or Ethernet at 1Gbps. You will need to swap SFPs (for now, at least) and reconfigure the port, and you must cycle/reset the module in which the ports reside in order to switch personality. (It’s my understanding that future improvements will require only cycling the specific port being switched.) Personally, I think the Unified Port functionality is extremely useful and I’m glad to see Cisco deliver it.
  • Cisco is also announced a new C-series rack mount server, the C260 M2. This is a dual-socket server running Intel Westmere CPUs, supporting up to 64 DIMM slots (using Cisco’s Memory Expansion technology) and supporting up to 16 HDDs or SSDs. It’s a pretty beefy server. From a virtualization perspective, the expanded memory footprint is pretty useful, but I’m not so sure that the 16 drive bays is much of a selling point. I’d love to hear others’ thoughts on that matter.
  • Also in the new product lineup are some new Catalyst 6500 modules, an ACE-30 module, an ASA service module, and a ES-40 40Gb DCI module for high-speed data center interconnects.
  • Finally, Cisco is launching an FCoE module for the MDS, which will allow customers to integrate FCoE into their networks while preserving their investment in the MDS infrastructure. On its own, this is interesting, but when coupled with other announcements…well, read on for more details.

However, the launch isn’t just about new products; if it were, it would be pretty boring, to be honest. The news included in the launch about the expansion of existing technologies is, in my mind, far more significant:

  • First, Cisco is stepping away from the “VN-Tag” moniker and focusing instead of the IEEE standardization of that functionality, a la 802.1Qbh. That’s a good move, in my opinion; they’re also going to be adding 802.1Qbg support in the future. Not such a good move (again, in my opinion) is the rebranding effort around VN-Tag and VN-Link, which are now going to be called Adapter FEX and VM-FEX. I don’t know that this new naming convention is going to be any less confusing or any more clear than the previous strategy. It is a bit more technically accurate, at least, since—if you read my post on network interface virtualization—you know that the Cisco Virtual Interface Controller (VIC), aka “Palo”, is really just a fabric extender (FEX) in a mezzanine card form factor. So, calling this Adapter FEX is a bit more accurate. VM-FEX….well, this still isn’t so clear. If any Cisco folks want to help clear things up, feel free to jump in by posting a comment to this post.
  • Cisco is adding FCoE support to the Nexus 7000 via the F1 line cards. This brings FCoE support directly to the Nexus 7000, which is a move that many expected. Along with the FCoE support, Cisco is also introducing the idea of a new type of virtual device context (VDC), the storage VDC. A storage VDC will process all FCoE traffic passing through the switch, in a completely separate fashion from LAN traffic (which would run in a separate LAN VDC). At least initially, Cisco will require the use of a storage VDC to use FCoE with the Nexus 7000; that might change over time. As with the introduction of the FCoE module for the MDS 9500, this news is interesting, but is really only useful in conjunction with other announcements. Specifically…(drum roll please)
  • In perhaps the biggest announcement of the event, Cisco is now supporting full multi-hop FCoE topologies. As I understand it, there will be a new release of NX-OS that will support the creation of VE_ports and enable multi-hop FCoE topologies (up to 7 hops supported). This functionality will exist not only on the Nexus 5000/5500, but also on the Nexus 7000 and on the FCoE module of the MDS 9500. As far as I know, this also means that any Nexus 2000 series fabric extenders that support FCoE will also now support multi-hop FCoE via their upstream Nexus switch. Cisco is going to require dedicated FCoE links between switches, at least at first, and as I mentioned earlier will require the use of a storage VDC on the Nexus 7000. I believe that this is probably the most significant announcement being made, and when taken together with other FCoE-related announcements like the FCoE module on the MDS 9500 and FCoE support on the Nexus 7000, opens up lots of new possibilities for FCoE and Unified Fabric in the data center. I, for one, am really excited to dive even deeper into the design considerations around multi-hop FCoE and Unified Fabric. Any network gearheads interested in doing a brain dump on multi-hop FCoE for me?
  • Equally important in the long-term but (apparently) not making a big impact immediately is LISP (Location ID/Separation Protocol). I’ve been talking LISP with Cisco for a while now (as I suspect many of you have), so it’s good to see the official announcement. Lots of people confuse the purpose/role of LISP when compared to OTV; they are both equally important but in very different ways. Further, LISP does not replace OTV (or vice versa). I’ll probably try my hand at a separate blog post to specifically discuss these two technologies and how the plan is for them to work hand-in-hand with each other for workload mobility. For now, it should suffice to say that OTV addresses Layer 2 connectivity between data centers while LISP helps the rest of the network more efficiently understand and adapt to the Layer 2 connectivity between data centers. Both are necessary.

There were a few other tidbits—like the ability to run up to 6 Nexus 1000V VSMs on the Nexus 1010, or support for the Virtual Security Gateway (VSG) on the Nexus 1010—but this covers the bulk of the information.

All in all, it’s exciting to see some of these technologies coming to light, and I’m really excited to see how the data center is going to evolve over the next couple of years. It’s a great time to be in this industry, especially if you’re a glutton for learning new technologies like me!

As always, I invite and encourage discussion, so if I’ve left something important out or if I’ve misrepresented something, please speak up in the comments (courteously and with full disclosure, where applicable). Thanks!

UPDATE: In response to information gathered via Twitter (I’m @scott_lowe), I’ve updated the information on the 5584UP/5596UP and Unified Ports functionality. Thanks for the great discussion!

Tags: , , , ,

19 comments

  1. Stuart Miniman’s avatar

    Scott,
    I believe you are mis-speaking on the Unified Ports. It is NOT different SFPs, but that the port can support (and switch between) FC and Ethernet (that’s even cooler, right?). I wrote about this here: http://wikibon.org/wiki/v/FCoE_sales_steadily_growing_as_Juniper_and_HP,_not_Brocade,_compete_with_Cisco
    On the FCoE, Nexus 2000 should be able to be part of a multi-hop, although they are not a hop themselves (lots of confusion on multi-hop). I did not realize that VDC was required, but we think it’s an excellent feature to help storage management teams (we wrote about this here http://wikibon.org/wiki/v/Cisco_Announcement_Review%3A_Assessing_New_Switch_Functionality_in_2011)
    I wrote (and did a video) about the launch too which is here (apologies for all of the links, but hopefully they are useful): http://wikibon.org/blog/cisco-is-ready-for-the-next-generation-network-are-you/
    Looking forward to hearing more about LISP, which we’ve known pairs with OTV for some interesting DCI scenarios.
    Cheers,
    @stu
    Wikibon Researcher/Analyst

  2. slowe’s avatar

    Stu, thanks for your comment.

    I’m unclear how a port could switch between FC and Ethernet without requiring different SFPs. It would seem to me that you’d need an entirely new kind of SFP that would run both Ethernet (including FCoE) and FCP. And what about cabling? For a single port to do either without a swap of SFPs, you’d need to standardize on optical fibre, which shoots cost effectiveness in the head. If I’m wrong, I’m wrong—but I can’t wrap my head around how you’re explaining it.

    Thanks!

  3. roamsedge’s avatar

    I’m sure you’re correct, Scott.

    Cisco says: “Each port on the Cisco Nexus 5596P switch is numbered, and groups of ports are numbered based on their function. The ports are numbered top to bottom and left to right. The 48 fixed ports support 8-, 4-, 2-, or 1-Gbps Fibre Channel transceivers and 1- or 10-Gigabit Ethernet transceivers.”

  4. Stuart Miniman’s avatar

    Scott,
    HP has had this functionality since last year (FlexFabric Virtual Connect built on QLogic technology) – they call it Flex ports. Cisco has Unified Ports and Juniper now has the same technology.
    10GbE and FC can support the same physical layer and software changes. It’s obviously not at the same time and varies by vendor as to how it changes – HP claims that you could leave a cable in the switch (wire once) and switch the other end from FC to Ethernet and it would switch. Cisco requires the port group to be rebooted (but once again, the switch cable could be left in).
    Hope this explanation helps – it is cool stuff that I’ve written about a few times over the last year.
    Stu

  5. Jon Langemak’s avatar

    Scott – You are entirely correct. There is no ‘magic’ SFP module that will do both FC 1,2,4,8 and E1,10. That’s just not possible due to line coding being different. So yes, the beauty of the switch is that you can upgrade at any time by switching SFP modules. Buy it now with 1 gig SFPs and next year pull them out and put 10 gig in. It’s a pretty neat deal. What blows me away is that you have close to a 2 Tbps switching fabric. Up to 96 10gig line rate ports. An accomplishment on its own.

  6. Fabio’s avatar

    Thanks for the insight Scott!!

    Looking forward to reading the LISP/OTV article :)

  7. Stuart Miniman’s avatar

    Scott,
    Well, looks like I need to make a retraction. Information that I have is that while it is NOT impossible for a single SFP to switch between FC and Ethernet (and some are publicly pitching this vision), currently shipping products are what you mention. Cisco needs to clarify the requirements to switch functionality – first you must switch SFP and what is the impact on port/module. Thanks for helping flesh this out.
    Stu

  8. Brian Gracely’s avatar

    Scott,

    Nice write up. Maybe the easiest way to think about the difference between Adapter-FEX and VM-FEX is this:

    Adapter-FEX is for non-virtualized environments where you want to extend the value of Virtual I/O to the OS and simplify the network topology & management

    VM-FEX (formerly known as “VN-Link in HW”) is for virtualized environment where you want to extend the value of Virtual I/O to the OS and simplify the network topology & management.

  9. Jon Langemak’s avatar

    Hi Stuart – Do you have a part number for a SFP that supports both FC and Ethernet on the 5596? Im finding it hard to believe that there’s a SFP that supports both Ethernet and native FC traffic on layer 1.

  10. EtherealMind’s avatar

    One small problem with LISP.

    Cisco seems keen to progress LISP as the definitive scaling protocol for the Internet and it seems they are extending its use within the Data Center here.

    However there is significant opposition from within the IETF Community and several competing options have forced a delay in the progess of any standard. I’ve blogged a little about it in this post http://etherealmind.com/path-lisp-certain-rfc-6115/ that refers to the RFC6115 which outlines the basis of the opposition and some Cisco engineers have responded in the comments so I won’t repeat them here.

    Because non-standardised approaches are equivalent to choosing to not have a health insurance policy for your network, the LISP approach should be chosen with due consideration. I think it’s good technology, just that industry hasn’t necessarily agreed it’s the best way forward and, to some extent, you are gambling on this technology in your data centre.

    greg

  11. Colby Cousens’s avatar

    Scott, thanks for the post. I’m excited about the C260 announcement; I’ve been putting together some specs for a VDI in a box scenario using a C250 M2 with SSD’s on board for quick disk performance. I can only get up to 100GB SSD’s from Cisco so I do see value in the increase to 16 slots if you’re trying to deploy high performance virtual desktops on a budget.

  12. Stuart Miniman’s avatar

    In response to Jon – no magic, see the Tweets from Cisco’s Omar Sultan confirming that this is a future coming soon:
    http://twitter.com/omarsultan/statuses/53505627070283776
    http://twitter.com/omarsultan/statuses/53509604310650880

  13. Omar Sultan’s avatar

    Scott:

    Thanks for the write-up. So, to clear things up a bit. On the topic of unified ports and switching FC Enet, the process is as follows:
    1. Change SPF
    2. Change port-type (2 commands)
    3. Bounce the GEM the port is part of. If the port is part on a GEM, bounce the GEM, if its on the baseboard, bounce the chassis. In an upcoming rev of sw, we will change this so you will just need to bounce the individual ports.

    As far as the re-naming around VIC and VN-Tag, they all use the same underlying tagging technology, so it seemed to make sense to fold them under common naming. The difference is where you end up terminating the link: at the port, at the adaptor, or at the VM.

    Hope that helps,

    Omar Sultan (@omarsultan)
    Cisco

  14. slowe’s avatar

    Omar, thanks for the clarification—I appreciate it!

    Greg, thanks for the additional information. I’ll have a look at the reference article to get a better idea of what’s going on around LISP.

    Colby, there are some that would say “high performance virtual desktops” and “on a budget” don’t mix. :-) I tend to consider shared storage before building out local storage, but I know that the argument of local storage vs. shared storage for VDI is a lively discussion and there are plenty of opinions in both directions. If you are considering local storage, then I agree that the C260 makes a lot of sense.

    Thanks to everyone for the great discussion!

  15. Jon Langemak’s avatar

    Stuart – Looks like Omar cleared it up in his post below. You still need different SFPs to handle different types of technology (Native FC, Ethernet).

    Thanks Omar!

  16. Steve Puuka’s avatar

    This is an exciting time in the data center development. Cisco, Juniper, Brocade and HP are all making moves to reduce latency and expand any-to-any port counts by creating fabrics.

    Check out the Packet Pushers podcast discussion of the recent trends in this area. Lots of fun ahead as these turn from concept to products.

    http://packetpushers.net/show-38-comparing-switch-fabrics-juniper-brocade-cisco/

  17. Erik’s avatar

    “Unified Ports are ports that can support either Fibre Channel (supporting up to 8Gbps FC), Fibre Channel over Ethernet (at 10Gbps), or Ethernet at 1Gbps.”

    Any idea why it doesnt support 10GigE (without FC)?

  18. slowe’s avatar

    Erik,

    I didn’t specifically call out 10GigE without FCoE, but it is fully supported.

  19. Stuart Miniman’s avatar

    Scott,
    I’m told that Cisco now supports the optics that can switch personalities between FC and Ethernet on unified ports.
    QLogic also announced a switch that will have the option for those optics: http://wikibon.org/wiki/v/FC_and_Ethernet_as_a_Single_Solution_Set

Comments are now closed.