bladevrack

By Aaron Delp

Welcome to Part 4 in our series on blades and virtualization. Please note that I will be covering the HP virtualized I/O offerings in a near future post. I plan to cover Virtual Connect, Flex-10, and the BL495 blade at that time.

As always, let’s start by reviewing the hardware specification we will be using to compare the rack servers against the blade servers. Here are the portions of the spec that concern us:

• A minimum of 32 GB memory – As many have commented and my own studies have shown, memory is usually the first limiting factor. The more memory you can install in the system, the better.
• 2 on-board NICs + some combination of expansion cards (FC, iSCSI, NIC, 10Gb)

As we did for the IBM side, let’s start with memory. The maximum amount of memory available on the DL360 is currently 64GB when using 8GB DIMMS in each of the eight slots. The BL460c has the same memory configuration (8 slots), but has a few caveats depending on the processor power consumption. According to the HP Product Bulletin Tool, if the processor is 80W or below, then the max amount of memory is 64GB (8x8GB). If you would like to use processors above 80W, then the maximum amount of memory is 48GB in a 6x8GB configuration. Which processors take above 80W you ask? As of this writing, there are only 2 processors in this category, the 3.0GHz Quad Core (Intel X5450 chip, not the E5450 chip!) and the 3.16GHz Quad Core (Intel X5460).

As with the IBM solution, I will explore four possible transport technologies for virtualization: 10Gb, iSCSI, FC, and NFS. I will also fill any remaining expansion slots with 1Gb Ethernet ports. Just like the IBM Chassis, we have a maximum of eight expansion ports.

HP expansion is a little more straightforward than IBM. Bays 1 & 2 connect to the blade on-board NICs and must be populated with an Ethernet compatible switch. An HP BL460c blade has two expansion slots, labeled Mezzanine 1 & 2. Bays 3 & 4 on the chassis connect to the adapter in Mezzanine Slot 1 on the BL460c Blade. Bays 5-8 connect to Mezzanine bay 2 on the blade. If a dual port card is placed in Mezzanine 2, only Bays 5 & 6 will be active. A four port card is required (Ethernet is the only 4 port card currently) to access all four switch bays.

The only exception to the above port mappings is 10Gb. Like the IBM 10Gb switch, the HP 10Gb switch is a “double wide” switch. For dual 10Gb in the HP chassis, one switch must be placed in Bays 5/6, the other in Bays 7/8.

Here are the possible configurations given the port mappings above:

BL460c blade with 10Gb & 4 NICs:

BL460c blade with 10Gb, 2 NICs, and 2 FC:

BL460c blade with FC & 6 NICs:

BL460c blade with iSCSI & 6 NICs:

BL460c blade 8 NICs:

As you can see from the tables above, HP has a nice round portfolio without any holes. In the next section we will be covering the IBM virtualized I/O solutions.

By Aaron Delp

UPDATE:  I have made some corrections to the 10GB and iSCSI sections based on feedback.  Thank you!

Welcome to part 3 of my series on blades and virtualization. I will be breaking the expansion articles into four parts: traditional IBM options, traditional HP options, virtualized IBM options, and virtualized HP options. By “virtualized options” I mean the IBM and HP blade I/O virtualization solutions (like IBM Blade Open Fabric Manager and HP VirtualConnect), not a server configuration that will run virtualization. Today, we will be covering IBM traditional options.

Let’s take a second to revisit a portion of our hardware standard we have been using for the power series and make a few additional notes. Here is the spec again:

• A minimum of 32 GB memory – As many have commented and my own studies have shown; memory is usually the first limiting factor. The more memory you can install in the system, the better.
• 2 on-board NICs + some combination of expansion cards (FC, iSCSI, NIC, 10Gb)
• 2 x 146GB, 10k SAS drives (except the IBM HS21XM blades which only support one HD)

I will get into the expansion options in a second, but let me start with the memory since this is the area most people are concerned about. With the recent addition of 8GB DIMMS to the HS21XM product line, the HS21XM now supports 64GB of memory. The IBM 3550 supports a max of 32GB and even the IBM 3650 only supports 48GB. If memory is your concern, the IBM blade is actually the BEST fit for virtualization.

Another area of concern is the fact that the HS21XM only supports one hard disk. If you would like to boot the OS from a local disk, this can be an issue. IBM also offers the dual solid state drive in a RAID-1 configuration but both platters are mounted in the same enclosure so if you lose one, you need to replace them both. We know because it happened to one of our customers. In the end that doesn’t make sense either. I don’t see the actual ESX build as critical in a VirtualCenter environment so I usually don’t worry about it excessively.

The expansion cards can get a little tricky depending on the configuration. For virtualization I see four possible options to fill out the blades to meet our needs. They are FC, 10Gb, iSCSI, and NFS. We will fill any remaining expansion with Ethernet 1GB ports to provide as much connectivity as possible.

Above is a picture of an IBM BladeCenter H Chassis from the rear, highlighting the expansion bays. Bays 1 and 2 connect to the onboard Ethernet so the switches must be Ethernet compatible. Bays 3 and 4 connect to the first expansion card on the IBM Blade, often called the CFFv form factor. The v in CFFv stands for “vertical” meaning this card will talk to the vertical expansion switches.

Bays 7-10 often cause much confusion.  They are positioned horizontally in the chassis and connect to any CFFh (h for horizontal, get it?) card on a blade. A number of different switches can be inserted into these bays. The first one is the easiest one, the 10Gb switch.  The first switch would go in bay 7.  The second would go in to bay 9.  Each switch would connect to a port on the dual port 10Gb card.

UPDATE: You can also do a 4x10GB configuration by adding 2 more 10GB switches in Bays 8 and 10.  You will then need the 4 port 10GB card instead of the 2 oprt 10GB card.  I have updated the charts below to reflect the MAXIMUM number of connections.  You can of course go with less.

The second option for Bays 7-10 involves something called the MSIM (Multi-Switch Interconnect Module). This option will allow the FC, iSCSI, and NFS configurations. The MSIM is an adapter that divides the High Speed Bays into Bays 7 to 10 shown above. To take advantage of all four bays, two MSIM’s divide both High Speed Bays and you will need a four port CFFh card to map to all four bays.

The above information yields the following switch and blade expansion card combinations:

Blade with 10Gb & 4 NICs (no MSIM’s – 4 NICs and 4 10Gb per blade):

Blade with 10Gb, FC & Eth (no MSIM’s – 2 NICs, 2 FC and 4 10Gb per blade):

Blade with FC Option #1 (requires 2 MSIM’s – 2 FC and 6 NICs per blade):

Blade with FC Option #2 (requires 2 MSIM’s – 2 FC and 6 NICs per blade):

Blade with NFS or software iSCSI (2 MSIM’s – 8 NICs per blade):

You will notice that only 3 of the 4 options are given. Hardware-based iSCSI is missing. That is because IBM has a hole in their portfolio. The IBM iSCSI card is the older form factor that if used on an HS21XM blade it doesn’t allow any other card. So, you end up with 2 NICs and 2 iSCSI. This isn’t acceptable and isn’t a valid option.

So what do you think? I believe that any of the above configurations will suit virtualization nicely given a proper design. I look forward to your thoughts as we continue on to the next article, HP traditional expansion.

By Aaron Delp

This is the second part of a series of articles on Blades and Virtualization. The first article is found here. Today, I will present IBM rack servers versus blade servers and explore the power consumption for each.

In case you need it again, here is the hardware platform I will be using as the standard:

• 2 x E5450 3.0 Ghz quad-core processors
• 32 GB memory in an 8 x 4GB configuration
• 2 on-board NICs
• Additional quad NIC card for a total of 6 NICs
• Dual FC card (assuming FC SAN storage for VMs but it could easily be NFS or iSCSI as well)
• 2 x 146GB, 10k SAS drives (except the IBM HS21XM blades which only support one HD)

Here is the link to the IBM System x and Blade Power Configuration Tool I used for the numbers below. Using the hardware specifications above, I will be comparing the IBM System x 3550 server against the IBM HS21XM blade server.

Let me start by giving you a little history of the IBM Power Tool and note some limitations that may skew these results slightly. In case you haven’t used it before, the IBM tool is a little rough around the edges. Not all of the products and configurations are included in the tool. The Power Tool was started as a side project by some members of the IBM Brand Team a few years ago and was picked up by IBM as a product. Although I knew the original authors before they moved on to other jobs, I am not sure who keeps it up to date today.

The System x 3550 calculations are pretty accurate but the IBM Blade Center calculations are missing a few pieces. The tool will only allow for up four switches in the chassis. Even though you can configure two MSIM’s to enable the additional switch bays, you can’t choose the additional switches in the tool. Also, the IBM Blades section of the tool won’t allow for the CFFh expansion card that would attach to the switches. So, these calculations are missing four switches per chassis and one expansion card per blade. Not a big deal but I wanted to point it out. Also, please remember that all calculations assume 100% utilization of the server(s).

With all of that out of the way, what does a fully loaded IBM chassis look like compared to rack servers? An IBM BladeCenter H Chassis holds fourteen servers so we will begin there:

 
As you can see, from a power perspective, the IBM blades are pretty impressive! We are looking at approximately 30% savings in all areas versus IBM rack servers. Let me be very clear on one point though: I am not stating that IBM blades are more power efficient than HP blades (I know that comparison is right around the corner for many of you). For many reasons that I won’t go into here, I don’t believe you can make a jump to that conclusion based on the numbers. I am stating that IBM blades are more efficient than IBM rack servers, nothing more.

How about a chassis that is half full with seven servers?

 
We are seeing 20% or greater with a half full chassis. Again, the numbers are very nice.

Lastly, what is the breakeven point? For the IBM blades the breakeven point is four servers.

 
The most interesting bit of information I find with these numbers is the difference the fourth blade makes. With three blades, you have slightly negative percentages (the rack servers are better). But, add the fourth blade and you are suddenly saving somewhere near ten percent. I find that odd.

My conclusion this time is a cut and paste of last time (almost literally!). As with any consolidation project, there is a breakeven point with blade servers where the initial investment begins to pay off. Also, like virtualization, the more blades (or virtual machines) you add to the environment, the more savings you will see.

The next article will focus on the expansion abilities of both the IBM and HP blade servers. I know that expandability is a big concern for many readers, so stay tuned for Part 3!