Hardware

You are currently browsing articles tagged Hardware.

Creating UCS Service Profiles Part 1: Networking Elements

Wednesday, May 5, 2010 in Networking, Virtualization by slowe | 2 comments

This article is part 1 of a series of posts that provide some additional information on creating UCS service profiles. In this part of the series, I provide some guidelines on networking-related elements that should be created before you create a UCS service profile. By creating these elements before creating the service profiles or service profile template, the process of creating the service profile or service profile template is a bit smoother.

There are three key networking elements that I recommend defining before you create a UCS service profile:

  1. At least one MAC address pool
  2. All the VLANs that need to be visible/accessible within UCS
  3. At least one vNIC template

The steps for creating each of these elements, along with their location in UCSM, is found below.

Creating a MAC Address Pool

Location in UCSM: LAN tab – LAN > Pools > MAC Pools
Prerequisites: None

Here are the steps for creating a MAC address pool:

  1. Right-click on MAC Pools and select Create MAC Pool.
  2. Specify a name and description for the new MAC pool, then click Next.
  3. Click Add (with the green plus symbol) to add a new MAC block.
  4. Specify the first MAC address and the size of the MAC block. Keep in mind that MAC pools (and the MAC address blocks within them) are not shared across UCSM instances, so be sure to provide a way of distinguishing MAC addresses across multiple UCSM instances. For example, use the fourth hextet to denote a UCSM instance (01 for the first UCSM instance, 02 for the second, etc.). This will help generate unique MAC addresses across multiple UCSM instances.
  5. Click OK to add the MAC address block to the MAC pool.
  6. Repeat steps 3 through 5, as needed, to add more MAC address blocks to this MAC pool. When the MAC address pool is complete, click Finish to complete the creation of the new MAC pool.

Defining VLANs

Location in UCSM: LAN tab – LAN > LAN Cloud > VLANs
Prerequisites: None

  1. Right-click on VLANs and select Create VLAN(s).
  2. Specify a descriptive name for the VLAN.
  3. Specify whether the VLAN should be global to both UCS fabrics, specific to only one fabric or the other, or if the fabrics should be configured differently for the same VLAN. Generally, VLANs will be configured as Common/Global.
  4. Specify the VLAN ID(s) that are associated with this VLAN object in UCSM. Generally, each VLAN object in UCSM should only have a single VLAN ID associated with it. If you enter multiple VLAN IDs (like “70-75″ or “21-24,27″), UCSM will create separate VLAN objects for each VLAN ID specified. This makes the creation of multiple VLANs a single step.
  5. When the dialog box is complete, click OK to create the VLAN object.
  6. Repeat this process to create VLAN objects for all VLANs that should be available within this UCSM instance. Use multiple VLAN IDs, as outlined earlier, to streamline the creation of multiple VLANs at the same time.

Creating a vNIC Template

Location in UCSM: LAN tab – LAN > Policies > vNIC Templates
Prerequisites: MAC address pool, VLANs

  1. Right-click on vNIC Templates and select Create vNIC Template.
  2. Specify a name and a description for this vNIC template.
  3. Select the UCS fabric to which this vNIC template should be associated. Do not select Enable Failover when using VMware ESX/ESXi. Note that you will need to create vNIC templates for both fabrics in dual fabric configurations.
  4. Leave all targets selected.
  5. Select Updating to make this an updating template. As additional VLANs are added in the future, modifying this template will then automatically modify service profiles created from this template to make the new VLANs available to associated blades.
  6. Place a check mark next to each VLAN that should be visible to vNICs created from this vNIC template.
  7. Mark one of the selected VLANs as the native VLAN; traffic from this VLAN will be untagged traffic. (In VMware ESX/ESXi, this traffic would require a port group with a blank VLAN ID configured.)
  8. Select the MAC address pool created earlier.
  9. If necessary, choose values for QoS policy, network control policy, pin group, and stats threshold policy. In general, these values do not need to be set.
  10. When the dialog box is complete, click OK to create the vNIC template.
  11. If this is a dual fabric installation, repeat this process for the second UCS fabric.

In part 2 of the series, I’ll provide information on the storage-related elements that I recommend creating before you create a UCS service profile.

If you have any questions or clarifications, please feel free to post them in the comments.

Tags: , , , ,

Another Collection of UCS Posts

Monday, April 19, 2010 in Virtualization by slowe | 1 comment

It’s been a couple of months since I posted my last collection of UCS-related posts. Since that time, I’ve collected a few more posts, so here’s another collection of UCS posts (in no particular order).

The State of Statelessness: Cisco UCS vs. HP Virtual Connect
Gestalt IT Tech Field Day – On Cisco and UCS
How Cisco UCS Deals with Split Brains
L2 Forwarding Rules in UCS EHV Mode
Scalability Study for Deploying VMware View on Cisco UCS and EMC Symmetrix V-Max Systems
Cisco C200M1 CIMC Update Process

As I uncover more UCS posts, I’ll bring them to your attention here. If you have some useful UCS information or useful UCS posts, please feel free to add them to the comments below (with full vendor disclosure, please). Thanks!

Tags: , ,

A Collection of UCS Posts

Thursday, February 18, 2010 in Virtualization by slowe | 2 comments

There’s been quite a few good Cisco UCS posts published recently; I thought it might be handy to collect a list of some of them (I’m sure that I will miss some). Here are a few that I’ve seen over the past few weeks (in no particular order):

Swapping UCS Blades with Local Boot Policies
Get Spidey Powers with UCS; but with Great Power comes Great Responsibility
25 ways that Cisco UCS frees you to do other things
Cisco UCS – How Many FEX Uplinks Do I Need?
UCS local disk policy + some vBlock
Cisco UCS: different workload, different configuration, same blade. Simple.
Cisco UCS Information for “Server People”
Cisco UCS vs. IBM and HP – Where are the Brains?
UCS Gotchas? and how much time does it take day to day?

Anyone else have any UCS posts that have surfaced recently? Add them in the comments below.

Tags: , ,

What is SR-IOV?

Wednesday, December 2, 2009 in Virtualization by slowe | 18 comments

I/O virtualization is a topic that has received a fair amount of attention recently, due in no small part to the attention given to Xsigo Systems after their participation in the Gestalt IT Tech Field Day. While Xsigo uses InfiniBand as their I/O virtualization mechanism, there are other I/O virtualization technologies out there as well. One of these technologies is Single Root I/O Virtualization (SR-IOV).

So what is SR-IOV? The short answer is that SR-IOV is a specification that allows a PCIe device to appear to be multiple separate physical PCIe devices. The SR-IOV specification was created and is maintained by the PCI SIG, with the idea that a standard specification will help promote interoperability.

SR-IOV works by introducing the idea of physical functions (PFs) and virtual functions (VFs). Physical functions (PFs) are full-featured PCIe functions; virtual functions (VFs) are “lightweight” functions that lack configuration resources. (I’ll explain why VFs lack these configuration resources shortly.)

SR-IOV requires support in the BIOS as well as in the operating system instance or hypervisor that is running on the hardware. Until very recently, I had been under the impression that SR-IOV was handled solely in hardware and did not require any software support; unfortunately, I was mistaken. Software support in the operating system instance or hypervisor is definitely required. To understand why, I must talk a bit more about PFs and VFs.

I mentioned earlier that PFs are full-featured PCIe functions; they are discovered, managed, and manipulated like any other PCIe device. PFs have full configuration resources, meaning that it’s possible to configure or control the PCIe device via the PF, and (of course) the PF has full ability to move data in and out of the device. VFs are similar to PFs but lack configuration resources; basically, they only have the ability to move data in and out. VFs can’t be configured, because that would change the underlying PF and thus all other VFs; configuration can only be done against the PF. Because VFs can’t be treated like a full PCIe device, then the OS or hypervisor instance must be aware that they are not full PCIe devices. Hence, OS or hypervisor support is required for SR-IOV so that the OS instance or hypervisor can properly detect and initialize PFs and VFs correctly and appropriately. At this time, SR-IOV support is only found in some of the open source Linux kernels; this means it will find its way into KVM and Xen first. I do not have a timeframe for SR-IOV support in VMware vSphere or Microsoft Hyper-V.

So, putting this all together: what do you get when you have an SR-IOV-enabled PCIe device in a system with the appropriate BIOS and hardware support and you’re running an OS instance or hypervisor with SR-IOV support? Basically, you get the ability for that PCIe device to present multiple instances of itself up to the OS instance or hypervisor. The number of virtual instances that can be presented depends upon the device.

The PCI SIG SR-IOV specification indicates that each device can have up to 256 VFs. Depending on the SR-IOV device in question and how it is made, it might present itself in a variety of ways. Consider these exampes:

  • A quad-port SR-IOV network interface card (NIC) presents itself as four devices, each with a single port. Each of these devices could have up to 256 VFs (single port NICs) for a theoretical total of 1,024 VFs. In this case, each VF would essentially represent a single NIC port.
  • A dual-port SR-IOV host bus adapter (HBA) presents itself as one device with two ports. With 256 VFs, this would result in 512 HBA ports spread across 256 dual-port virtual HBAs.

These are, of course, theoretical maximums. Because each VF requires actual hardware resources, practical limits are much lower. Currently, 64 VFs seems to be the upper limit for most devices.

In situations where VFs represent additional NIC ports or HBA ports, other technologies must also come into play. For example, suppose that you had an SR-IOV-enabled Fibre Channel HBA in a system; that HBA could present itself as multiple, separate HBAs. Of course, because these logical HBAs would still share a single physical HBA port, you’d need NPIV (more information here) to support running multiple WWNs and N_Port_IDs on a single physical HBA port.

Similarly, you might have a Gigabit Ethernet NIC with SR-IOV support. That NIC could theoretically (according to the PCI SIG SR-IOV specification) present itself as up to 256 virtual NICs. Each of these NICs would be discrete and separate to the OS instance or hypervisor, but the physical Ethernet switch wouldn’t be aware of the VFs. Switches wouldn’t, by default, reflect some types of traffic arriving inbound on a port (from one VF) back out on the same port (to another VF). This could create some unexpected results.

SR-IOV does have its limitations. The VFs have to be the same type of device as the PF; you couldn’t, for example, have VFs that presented themselves as one type of device when the PF presented itself as a different type of device. Also, recall from earlier that VFs generally can’t be used to configure the actual physical device, although the extent to which this is true depends upon the implementation. The SR-IOV specification allows some leeway in the actual implementation; this leeway means that some SR-IOV-enabled NICs may also have VF switching functionality present (where the NIC could switch traffic between VFs without the assistance of a physical switch) while other NICs may not have VF switching functionality present (in which case VFs would not be able to communicate with each other without the presence of a physical switch).

I do want to point out that SR-IOV is related to, but not the same as, hypervisor bypass (think VMDirectPath in VMware vSphere). SR-IOV enables hypervisor bypass by providing the ability for VMs to attach to a VF and share a single physical NIC. However, the use of SR-IOV does not automatically indicate the hypervisor bypass will also be involved. Hypervisor bypass is a topic that I’m sure I will discuss in more detail in the near future.

Finally, it’s worth noting that the PCI SIG is also working on a separate IOV specification that allows multiple systems to share PCIe devices. This specification, known as Multi-Root IOV (MR-IOV), would enable multiple systems to share PCIe VFs. I hope to have more information on MR-IOV in the near future as well.

You now should have a basic understanding of SR-IOV, what it does, what is necessary to support it, and some of the benefits and drawbacks that SR-IOV creates. Feel free to post any questions you have about SR-IOV in the comments and I’ll do my best to get answers for you.

Tags: , , , ,

Getting QLogic Gen2 CNAs Working with ESX 4

Thursday, August 13, 2009 in Storage, Virtualization by slowe | 8 comments

If you’ve been following my updates on Twitter, you know that I’m in the process of installing and configuring a Nexus 5000 switch. The Nexus 5010 will provide 10Gb Ethernet and Fibre Channel over Ethernet (FCoE) to the servers and storage arrays in the lab, and I’ll be using the equipment to help generate reference architectures and standard configurations for the ePlus virtualization team.

One of our storage partners sent over some second-generation (Gen2) Qlogic QLE8142 dual-port converged network adapters (CNAs) for me to use. These adapters joined the Emulex LP21000 CNAs that I had already procured for the lab, and I’d use the Qlogic and Emulex CNAs to connect my HP DL385G2 servers to the Nexus 5010. I don’t know if it’s because these cards are cutting edge (they are pre-production/beta units, apparently), but if all Nexus and FCoE deployments are this difficult to get up and running then FCoE will have a bleak future indeed. I’ve managed to finally get them working and recognized, but it most certainly was not as straightforward as I had anticipated.

Here’s what I’ve had to do to make them work (with thanks to Eric H for his help):

  1. I flashed the HP DL385G2 to the latest firmware available from HP. I don’t know if this is absolutely necessary (I’ll find out when I do the next server), but I figured it couldn’t hurt.
  2. Next I had to obtain a special beta flash image of the firmware and BIOS for the CNA and flash it to the latest version. The first time I flashed it, the CNA reported errors during the POST on the next reboot. I repeated the process and the errors went away.
  3. The in-box drivers for VMware ESX 4.0.0 don’t support the Gen2 QLogic cards, so I had to obtain beta drivers to install into VMware ESX. I first tried esxupdate with the bundle ZIP file, like this:
    esxupdate update --bundle=offline-bundle.zip --nosigcheck
    I couldn’t omit the --nosigcheck parameter because esxupdate then failed with error 20 (VibSigMissing). That, unfortunately, worked only for the Ethernet drivers and not the HBA drivers. To make the HBA drivers install, I had to install the RPM directly and include the --force parameter because ESX thought the version that was already installed was newer than the beta driver I was trying to install. I did finally get both sets of drivers installed, though.

Once both sets of drivers were installed, the 10GbE NICs and the 10Gb HBAs were both recognized in vCenter Server, but the ESX console showed an error about missing signatures. You can see a screenshot of the error here.

I’m quite confident that this is just because this card is an early Gen2 card, and that the strangeness I’ve been seeing is not a reflection on QLogic or VMware. I am a bit disappointed that the CNA is not supported by ESX 4.0.0 out of the box; I guess that will probably be remedied in ESX 4.0 Update 1. (No, I don’t have any special insider knowledge. I’m just guessing.)

Even so, I wanted to share this information with the readers so they could be prepared in case they are planning an FCoE deployment. Just make sure that you are prepared to update the BIOS and firmware of your server and potentially your CNAs, and be prepared to install custom drivers. Note that I’m going to try a fresh install of ESX 4 on this system and install the drivers during the installation process to see if that helps to correct the errors I’ve seen thus far. I’ll post an update to this article when I have more information.

UPDATE: Flashing the HP DL385G2 BIOS isn’t necessary; after posting this article, I tried the same procedure with a server using an older BIOS. The card worked fine. Also, I made a major mistake in the original version of this article—the cards are actually QLogic cards, not Emulex cards. The Emulex LP21000 CNAs that I have are Gen1 CNAs and, aside from a strange cabling issue I’m chasing, appear to work just fine.

Tags: , , , , , ,

Netronome and I/O Virtualization

Tuesday, August 4, 2009 in Networking, Virtualization by slowe | 2 comments

At what point does a product become a virtualization product? With all the hype and the buzz around virtualization—predominantly because of VMware’s success—every company is hopping on the “virtualization bandwagon” and releasing a new “virtualization product”. In most cases, these products are simply repackaged versions of their existing products, perhaps with a slight improvement here or there to justify the virtualization moniker.

Of course, virtualization is more than just machine virtualization such as that offered by VMware, Microsoft Hyper-V, and Citrix XenServer. Virtualization is about inserting layers of abstraction between different computing resources, and there are other valid forms of virtualization. Take application virtualization, for example. Products such as ThinApp and App-V insert a layer of abstraction between applications and the underlying operating system.

A couple of weeks ago I had the opportunity to speak with Netronome, a small company developing a 40-core processor called the Network Flow Processor (NFP). Based on technology acquired from Intel, Netronome touts the NFP as an I/O virtualization product. But is this another instance of repackaging an existing product with the virtualization moniker just to capitalize on a sales opportunity, or is the NFP truly an I/O virtualization product? Like a lot of other products, it’s really hard to say.

Netronome’s upcoming product, the NFP-3200, is a hardware product that they intend to OEM to leading server manufacturers. The idea is that the NFP will be embedded into the server design to allow networking I/O functions to be offloaded to the NFP. Much ado has been made about hypervisor bypass (aka VMDirectPath), but rather than removing the software switch, why not put the switch into hardware? That would be one such function offered by the NFP. The NFP could provide line-rate switching between physical ports, virtual NICs, or any combination of the two. This approach with the NFP-3200 is almost like the best of both worlds—no “hypervisor overhead” for software switching but all the benefits of a local switch in each host.

Of course, the Netronome folks are also talking about all kinds of other functionality made possible by the NFP-3200, like flow classification, firewalling, load balancing, etc., all running at line speed inside the NFP. Of course, the keys are a) getting one or more major server manufacturers on board with the idea; and b) getting one or more hypervisor vendors on board with the idea.

Netronome wasn’t willing to share any information on the former (naturally), but with regard to the latter they did indicate that much of this functionality will be available on open source Xen almost immediately upon the release of the NFP-3200. That’s nice, but it doesn’t really accomplish a whole lot. They need to get VMware and Microsoft (more VMware than Microsoft right now) to jump on this idea.

Aside from the lack of widespread hypervisor support, my only other complaint with Netronome’s approach is that it isn’t standards-based. Rather than leveraging SR-IOV, the PCI SIG standard for PCIe-based device virtualization, Netronome is leveraging what they call Software Configurable Virtual Functions (SCVF). I understand the benefits they see out of using SCVF instead of relying upon SR-IOV, but I’d still prefer to see more innovation upon standards instead of innovating without standards. But what do I know? It might be that PCI SIG adopts SCVF as their future standard. Stranger things have happened…

Anyway, for more information visit the Netronome web site. If they can pull off support from one or more major server manufacturers and one or more major hypervisor vendors, the NFP-3200 could make the virtualization scene very interesting indeed.

Tags: , ,

Correction: There Might be an End-to-End FCoE Solution

Thursday, July 30, 2009 in Networking, Storage by slowe | 12 comments

This post is a follow-up from my post yesterday titled “No Such Thing as an End-to-End FCoE Solution”.

After publishing that post, I managed to get in touch with some very smart people who were willing to spend some time with me and educate me on the various intricacies involved here. In order to help you, my readers, understand the various pieces and parts, I’ll need to first provide some definitions.

Fibre Channel Forwarder (FCF): In its simplest form, this is another form for an FCoE switch. A Nexus 5000 would be an example of an FCF.

Multi-hop FCoE: There are a couple of different definitions here. One definition would be having multiple FCFs connected together (i.e., a Nexus 5000 connected to another Nexus 5000). A second definition would be having multiple Layer 2 hops between an FCoE initiator or target and an FCF. Note that the switches handling those hops must be IEEE DCB capable.

FCoE Initialization Protocol: FIP, as its more commonly known, was included in the FC-BB-5 FCoE standard that was finalized in early June.

OK, now that I have some definitions in place, I can discuss how it might be possible (eventually) to build an end-to-end FCoE solution.

Disclaimer: I don’t claim to be an FCoE expert, I’m just trying to understand it better myself and help others understand it better. If I’m misrepresenting something, let me know—courteously and professionally—in the comments, or drop me an e-mail.

The question “Can I build an end-to-end FCoE solution?” has multiple answers:

  • If you have only a single FCF and everything is plugged into that FCF, then you can build a pure FCoE solution today. The Nexus 5000 can function as the FCF, and both the CNAs and targets that are available will work. Obviously, this is not a very scalable solution.
  • If you have multiple FCFs, or if you have multiple Layer 2 hops between initiators or targets and the FCFs, then you might or might not be able to build an end-to-end FCoE solution. In this scenario, FIP-enabled initiators and targets would be able to find and communicate with each other, but non-FIP-enabled initiators and targets would not (unless they were plugged into the same FCF). At this point I am unclear about connectivity between pre-FIP initiators or targets on the same IEEE DCB-capable Layer 2 switch (not an FCF); I suspect they would not be able to communicate.

All of these statements are applicable until you bring UCS into the mix. For UCS, my earlier statement stands: with UCS, you cannot have an end-to-end FCoE solution today. That will change at some point in the future, but no one has shared any information with me regarding just how far in the future that might be.

If you have a pure Nexus 5000 environment, with FCoE-capable storage and servers with CNAs, you’d probably be able to make it work. With FIP support in that environment, you’d definitely be able to make it work. When you add UCS, though, it becomes very different. I hope to be able to discuss that in greater detail in the near future.

So, my earlier statement wasn’t entirely true; it is possible to build an end-to-end FCoE solution. Today, that solution would be very limited in size; once FIP support is baked into the initiators, the targets, and the FCFs, then the solution size will be able to scale.

As always, comments and clarifications are welcome!

Tags: , , , , ,

UCS Training, Day 1 Wrap-Up

Monday, July 27, 2009 in Networking, Virtualization by slowe | 3 comments

The first day of Cisco UCS training here in San Jose, CA, has just wrapped up. The class is providing some useful information so far, but I’m eager to delve deeper. I really hope the material deepens as the course progresses.

Aside from the northbound FCoE issue I posted about earlier today, there wasn’t anything too terribly surprising discussed so far. I was able to have a few questions answered. For example, the half-width blades don’t use Cisco’s advanced memory technologies and, therefore, will suffer from the same drop in memory transaction speed (MTS) as DIMM slots are populated—just like any other vendors’ Xeon 5500-based servers. I had kind of expected that Cisco would address that as a further point of differentiation from other vendors.

A few other points that came out of today’s discussion that might be useful to readers included:

  • Although the Cisco guys mentioned the use of Solid State Disks (SSDs) for the B-series blades, there’s no orderable option for SSDs yet.
  • Customers must buy RAM and disks for the B-series blades (and I would assume the C-series rack mount servers) from Cisco. There will be no support from TAC otherwise. I suspect this is an issue that will get squished pretty quickly; other server vendors have tried this approach and run afoul of US anti-trust regulations, at least with regard to memory. Remember Compaq telling customers they could only buy Compaq memory, or their warranty was void? I see something similar here.
  • When connecting the IO module to the fabric interconnect, you can use 1 uplink, 2 uplinks, or 4 uplinks—but you can’t use 3 uplinks. (File this under “strange limitations”.)
  • The number of virtual NICs (vNICs) or virtual HBAs (vHBAs) that can be presented from a Palo adapter is directly dependent upon the number of uplinks from the IO module in the blade chassis to the fabric interconnect. The stated maximum for Palo is 128 adapters, I believe; in order to get that many virtual adapters, you’d have to have multiple uplinks from the chassis to the fabric interconnects. I don’t have exact figures yet, but I’ll see if I can dig anything up.

I guess that’s about it for now; I’ll post more throughout the week. Thanks for reading, and keep the good comments coming!

Tags: , , , , ,

Potential UCS Issue: Northbound FCoE Connectivity

Monday, July 27, 2009 in Gestalt, Networking, Virtualization by slowe | 22 comments

I’m about halfway through the first day of Unified Computing System (UCS) training in San Jose, CA, and I’ve learned of what I think is a fairly significant limitation. The issue centers around what Cisco refers to as “northbound” traffic and how Fibre Channel over Ethernet (FCoE) is handled with northbound traffic.

Recall that a central part of UCS is the UCS 6100 series fabric interconnect. The 6100 series fabric interconnect has connectivity in two directions:

  • Southbound connectivity is connectivity aimed back at the fabric extenders in the blade chassis themselves.
  • Northbound connectivity is connectivity headed outside the UCS to other systems and networks.

All southbound traffic is 10Gbps Ethernet with FCoE. Northbound traffic can be 10Gbps Ethernet or Fibre Channel, but not FCoE. Based on the information I’ve been given (and if I’m incorrect please let me know in the comments), you cannot directly connect an FCoE-enabled storage array to a UCS. Even if your storage array has native FCoE interfaces, you can’t plug them into the UCS 6100 series fabric interconnects because that’s considered northbound traffic and you can’t use FCoE with northbound traffic.

I have a feeling customers who have purchased storage arrays with FCoE interfaces with the intention of hooking the arrays up directly to a UCS are going to be a bit upset when this information becomes more widely known.

If I’m working from incorrect or incomplete information, please feel free to speak up in the comments.

Tags: , , , , ,

Mudslinging Between Cisco and HP

Thursday, July 16, 2009 in General, Networking, PASS by slowe | 25 comments

I’ll preface this article by saying that I am not (yet) an expert with Cisco’s Unified Computing System (UCS), so if I have incorrect information I’m certainly open to clarification. Some would also accuse me of being a UCS-hater, since I had the audacity to call UCS a blade server (the horror!). Truth is, I’m on the side of the customer, and as we all know there is no such thing as a “one size fits all” solution. Cisco can’t provide one, and HP can’t provide one.

The mudslinging that I’m talking about is taking place between Steve Chambers (formerly with VMware, now with Cisco) and HP. HP published a page with a list of reasons why Cisco UCS should be dismissed, and Steve responded on his personal blog. Here are the links to the pages in question:

The Real Story about Cisco’s “One Giant Switch” view of the Datacenter (this was based, in part at least, on the next link)
Buyer beware of the “one giant switch” data center network model
HP on the run

I thought I might take a few points from these differing perspectives and try to call out some mudslinging that’s occurring on both sides. To be fair, Steve states in the comments to his article that it was intended to be entertaining and light-hearted, so please keep that in mind.

Point #1: Complexity

The reality of these solutions is that they are both equally complex, just in different ways. HP’s BladeSystem Matrix uses reasonably well-understood and mature technologies, while Cisco UCS uses newer technologies that aren’t as widely understood. This is not a knock against either; as I’ve said before in many other contexts and many other situations, there are advantages and disadvantages to every approach. HP’s advantage is that leverages the knowledge and experience that people have with their existing technologies: StorageWorks storage solutions, ProLiant blades, ProCurve networking, and HP software. The disadvantage is that HP is still tied to the same “legacy” technologies.

In building UCS, Cisco’s advantage is that the solution uses the latest technologies (including some that are still Cisco-proprietary) and doesn’t have any ties to “legacy” technologies. The disadvantage, naturally, is that this technological leap creates additional perceived complexity because people have to learn the new technologies embedded within UCS.

Adding to the simple fact that both of these solutions are equally complex in different ways is the fact that you must re-architect your storage in order to gain the full advantage of either solution. To get the full benefit of both UCS and HP BladeSystem Matrix, you need to be doing boot-from-SAN. (Clearly, this doesn’t apply to virtualized systems, but both Cisco and HP are touting their solutions as equally applicable to non-virtualized workloads.) This is a fact that, in my opinion, has been significantly understated.

Neither HP nor Cisco really have the right to proclaim their solution is less complex than the other. Both solutions are complex in their own ways.

Point #2: Standards-Based vs. Proprietary

Again, neither HP nor Cisco really have any room to throw the rock labeled “Proprietary”. Both solutions have their own measure of vendor lock-in. HP is right; you can’t put an HP blade or an IBM blade into a Cisco UCS chassis. Steve Chambers is right; you can’t put a Dell blade or a Cisco blade server into an HP chassis. The reality, folks, is that every vendor’s solution is has a certain amount of vendor lock-in. Does VMware vSphere have vendor lock-in? Sure, but so does Hyper-V and Citrix XenServer. Does Microsoft Windows have vendor lock-in? Of course, but so does…so does…well, you get the idea.

HP says VNTag is proprietary and won’t even work with some of Cisco’s own switches. OK, let’s talk proprietary…does Flex-10 work with other vendor’s switches? The fact of the matter is that both Cisco and HP have their own forms of vendor lock-in and neither can cry “foul” on the other. It’s a draw.

Point #3: The “Giant Network Switch”

At one point in HP’s article (I believe it was under the Complexity heading) they make this point about the network traffic in a Cisco UCS environment:

In Cisco’s one-giant-switch model, all traffic must travel over a physical wire to a physical switch for every operation. Consequently, it appears that traffic even between two virtual servers running next to each other on the same physical would have to traverse the network, making an elaborate “hairpin turn” within the physical switch, only to traverse the network again before reaching the other virtual server on the same physical machine. Return traffic (or a “response” from the second virtual machine) would have to do the same. Each of these packet traversals logically accounts for multiple interrupts, data copies and delays for your multi-core processor.

I do have to call “partial FUD” on this one. In a virtualized environment, even a virtualized environment running the Cisco Nexus 1000V, traffic from one virtual server to another virtual server on the same host never leaves that host. HP’s statement seems to imply that’s not the case, and as far as I know it is. However, HP’s statement is partially true: traffic from one virtual server on one physical host does have to travel to the fabric interconnect and then back again in order to communicate with a virtual server running on a physical host in the same chassis. The fabric extenders don’t provide any switching functionality; that all occurs in the interconnect. Based on the information I’ve seen thus far, I would say that using Cisco’s SR-IOV-based “Palo” adapter and attaching VMs directly to a virtual PCIe NIC would put you into the situation HP is describing, which then just reinforces a question that Brad Hedlund and I tossed back and forth a couple of times: is hypervisor-bypass, aka VMDirectPath, with “Palo” the right design for all environments? In my opinion, no—I again go back to my statement that there is no “one size fits all” solution. And considering that the use of hypervisor-bypass with “Palo” would put you into a situation where traffic between two virtual machines on the same physical host has to travel to the fabric interconnect and back again, I’m even less inclined to use that architecture.

In the end, it’s pretty clear to me that both HP and Cisco have some advantages and disadvantages to their respective solutions, and neither vendor really has the room to label the other as “more complex” or “more proprietary” than the other. But what do you think? Do you agree or disagree? Courteous comments (with full vendor disclosure) are welcome.

Tags: , , , ,

« Older entries § Newer entries »