January 2008

A friend and colleague of mine recently launched a blog of his own, The Blade Blog.  If the useful information that’s been posted there so far is any indication, this will be a definite addition to your RSS subscriptions.  Keep up the good work, Aaron!

UPDATE:  After some great feedback on the launch of his site, Aaron’s moved away from Blogger to a hosted WordPress solution and has gotten a new domain name.  You can now find Aaron at the Blade Vault.  More information and updated URLs available here and here.

SearchVMware.com recently published an article of mine about using HP VirtualConnect with ESX Server and the impact it has on configuring ESX Server networking.  Based on the comments to my original post about the article, it appears that I didn’t adequately explain the interaction between VirtualConnect’s Standard Ethernet Networks and VLAN tagging.  I’d like to thank those readers that asked about this issue and take a moment to clarify.

Standard Ethernet Networks are defined in the HP VirtualConnect Manager and allow you to control the mapping between physical NIC ports on the blades (the “downstream ports”) and the connections from the VirtualConnect switches to the external network infrastructure (the “upstream ports”).  For example, I could define a Standard Ethernet Network called “Production” and specify that Production would uplink either via Slot 2 Port 7 or Slot 2 Port 8.

That part is pretty straightforward.  Here’s how it plays into VLAN tagging with ESX Server.  Most organizations prefer to use VST (Virtual Switch Tagging; described in more detail in this article).  To use VST, the ports on the network infrastructure must be configured as 802.1Q VLAN trunks so that the external physical switches will pass the VLAN tags to the vSwitches inside ESX Server, where ESX Server can then handle the traffic appropriately.

To use VST with VirtualConnect and Standard Ethernet Networks, there is no difference.  The VirtualConnect upstream ports, as defined in the Standard Ethernet Network configuration, must be plugged into physical switch ports that are configured as 802.1Q VLAN trunks.  When you plug the upstream port into an 802.1Q VLAN trunk, the downstream port—the port going to the ESX Server—automatically becomes an 802.1Q VLAN trunk as well.  The VLAN tags will pass all the way to the ESX Server’s vSwitches, where ESX Server can deal with the traffic appropriately.

Likewise, if the VirtualConnect’s upstream ports are plugged into a static access port—a port which is not configured as an 802.1Q VLAN trunk but instead carries traffic only for a single VLAN—then the downstream ports also become static access ports and you are, effectively, using External Switch Tagging (EST).

I stated this in the original article in the third paragraph in the section titled “How Virtual Connect differs in ESX”:

In either way, these Ethernet networks will “pass through” the 802.1Q status of the physical switch port to which it is uplinked. If the physical switch port to which they are connected is configured as an 802.1Q VLAN trunk, then the downstream ports will act as 802.1Q VLAN trunks. Likewise, if the uplink is connected to a switch port that is configured as a static access port, then the downstream ports will act as static access ports.

Shared Uplink Sets, on the other hand, are different.  They don’t behave in the same way as Standard Ethernet Networks.  With Shared Uplink Sets, you are forced to use EST because the VLAN tags are stripped away at the VirtualConnect level.  The associated networks that are defined in VirtualConnect Manager define the different VLANs, and each downstream port is connected to an associated network.  Unlike with Standard Ethernet Networks, no VLAN tags are passed up to the ESX Server with Shared Uplink Sets.

So, to summarize:

• Standard Ethernet Network uplink connected to external switch port configured as 802.1Q VLAN trunk allows ESX Server to see VLAN tags and supports both VST and Virtual Guest Tagging (VGT)
• Standard Ethernet Network uplink connected to external switch port configured as static access port only allows EST configuration
• Shared Uplink Set only allows EST configuration

I hope this clarifies the interaction between HP VirtualConnect and ESX Server networking.  I welcome any further questions or clarifications in the comments below.  Thanks!

Momentum seems to be building around the Xen-based virtualization offerings from Citrix, as more ISVs announce support for Citrix XenServer or Citrix XenDesktop.  The latest announcements include:

SteelEye Announces Support for Citrix XenServer
Catbird Announces V-Security Beta for Citrix XenServer
Reflex Security Announces Reflex VSA Support for Citrix XenServer
VMLogix Introduces First Virtual Lab Automation Solution for Citrix XenServer

(Thanks to David Marshall of VMblog.com for the announcements.)

Citrix is also seeking to capitalize on the “Xen” name by renaming Presentation Server to XenApp.  Some have speculated that this means the product will be reshaped around and built on top of the Xen hypervisor.  Whether this is true or not is yet to be determined.

These recent announcements join the Microsoft announcement that Hyper-V will be able to use Citrix XenDesktop as a VDI broker and that SCVMM will be able to manage XenServer.

Taken together, these announcements show that the Citrix Xen-based offerings are gearing up for a serious offensive against the incumbent VMware in the virtualization space.  Of course, VMware isn’t standing still; VDM—Virtual Desktop Manager—has been officially released and a beta of VMware Stage Manager is also available.  (Thanks to Mike Laverick and Martin MacLeod for the links.)

Interesting times are definitely ahead.

I just finished reading the book titled Xen Virtualization: From Technologies to Solutions, published by Packt Publishing and written by Prabhakar Chaganti.  Overall, the book was helpful in getting up to speed with Xen, although it appears that English was not the native language for either Mr. Chaganti, his editor, or both.

The book is subtitled “A fast and practical guide to supporting multiple operating systems with the Xen hypervisor,” and it does live up to that subtitle.  The book very quickly moves into some hands-on exercises using a Linux host and the open source Xen hypervisor.  The exercises are fairly pertinent to the topic being discussed, and I especially liked the “What just happened?” sections after each hands-on procedure.  In those sections, the author breaks down the steps, the intended results, and the reasoning behind the procedure.  In my view, that’s a very helpful way to build understanding of the product.

My only complaint is—as I mentioned earlier—that English appears not to have been the native language for the author and/or editors.  The wording sometimes gets in the way of the content, making it more difficult than it should be to understand what the author is trying to say.  I would also say that I don’t think the book is worth the $40USD price tag that marked on the back.  At only 127 pages, $40 seems a bit steep.

Those issues aside, I found the book to be helpful in understanding Xen and some of Xen’s concepts.  I wouldn’t necessarily recommend this book to people who are both new to virtualization as well as new to Linux, as the material assumes a certain level of knowledge and experience with Linux.  Otherwise, if you have some Linux experience and want to get started with Xen, this book would be a good place to start.  (Just try to find the book on sale.)

As I’m sure most of you have already seen elsewhere, VMware has released version 1.1.1 of VMware Fusion, their desktop virtualization solution for Mac OS X.  I’ve already downloaded and installed the update, which is free for registered owners.

I won’t rehash all the feature lists and new features and such, as I’m sure that plenty of other sites have already reported that in excruciating detail.

I’ve not had the opportunity to really put it through its paces—that will happen tomorrow at the office—but I am happy to report that suspending and resuming virtual machines is much, much faster with this new release.  Since that’s primarily how I work, this improvement in speed is a real benefit to me.  For others, who don’t use the suspend/resume functionality as much, it may not be as beneficial.

You can download the Fusion update from VMware’s web site.

Via Ben Rockwood, I found this and thought it was just too funny to pass up:

It even has its own Amazon page.  You can view an online version here.

For some good information on using HP VirtualConnect with VMware ESX Server, go check out this article just published by SearchVMware.com:

While the idea of server profiles can be useful in VMware Infrastructure 3 (VI3) environments, Virtual Connect’s ability to integrate with VLAN tagging configurations is perhaps more applicable to ESX Server deployments on c-Class blades. In this article, we’ll take a look at how VirtualConnect integrates with ESX Server with regards to networking configurations.

Of course, I’m a bit partial to this particular author (wink, wink), but it is a good article nevertheless.

Feel free to hit me up with any corrections, comments, or thoughts.  And be sure to go read my article, so TechTarget will keep asking me to write for them!

The past couple of weeks have been very eventful, but things have finally sorted themselves out and I wanted to share the events with readers here.  A few weeks ago, I was approached by a well-respected individual in the storage industry about joining a major storage vendor.  I guess it’s OK to mention who it was: Network Appliance.

As you probably know, NetApp is a great place to work—voted #6 on the “Best Places to Work” list last year, if I recall correctly.  NetApp has a great environment, great people, an awesome benefits package…it’s a great opportunity all the way around.  I initially accepted the offer, but my current employer—ePlus Technology—then put forth a counter-offer and I was faced with choosing between two great organizations.  It was a difficult position to be in because I have a great deal of respect for both Network Appliance as well as ePlus.

In the end, I selected to remain with ePlus, but to move into a new role within the organization.  I won’t be spending as much time in the field with customers, although I will still be involved with some projects.  I’ll be focusing almost exclusively on and serving as a technical lead for virtualization technologies and related products.  This is very similar to what I would have been doing at Network Appliance, but by staying with ePlus I’ll get to include other technologies such as HP VirtualConnect, Ardence (oops, Citrix Provisioning Server), and eventually other virtualization platforms like Virtual Iron, Citrix/Xen, and Microsoft Hyper-V.  For a technology geek like me, it’s a great opportunity to get my hands on all kinds of different products and get to know them really well.  I also enjoy teaching—I was a Microsoft Certified Trainer in a past life—and there will be some of that involved as well.

All in all, I’m excited about the opportunity before me.  I strongly suspect that this new role will mean even more technical content here than in the past.

One final note to those at Network Appliance with whom I spoke during this process: You guys are a great group of people building a fantastic team.  If there is anything I can do for any of you, please don’t hesitate to let me know.  You know how to get in touch with me!

Over the last few weeks, I’ve been collecting various virtualization-related links in NetNewsWire’s Flagged Items collection, with the intention of blogging about them, bookmarking them, or both.  With time a bit short recently—let’s just say that life is really, really busy right now—I decided to just condense a bunch of them here with a brief commentary, where applicable, for each.  Hopefully some of this information will prove useful to some readers here.

• ESX Host Currently Has No Management Network Redundancy Error:  This is new to ESX Server 3.5; VMware HA reports a warning when it detects that there is no redundancy for the Service Console.  Clearly, this is an attempt to prevent situations where isolation response kicks in, and as the author points out can be mitigated by adding another NIC to the vSwitch where the Service Console port group is located.  I have also found that creating a second Service Console port group on another vSwitch will also remove the warning.  Duncan of Yellow Bricks also goes into more detail on Service Console redundancy on his blog as well.
• ESX “Configuring for HA” errors - What to do?:  VMware HA continues to be a sore spot, as Rick Vanover discusses here.  One useful tidbit of information from this article is the suggestion to go directly to the VPX_EVENT table of the VirtualCenter database to look for troubleshooting information.  Rick’s right—VirtualCenter’s error messages with regards to VMware HA are often totally useless.
• How to Use the Remote Command-line Interface to Invoke Storage Vmotion in Windows Server or Desktop:  Jack’s off to a great start to his blog at VMware World with a lot of very relevant and very useful information.  This article on using the RCLI to do Storage VMotion can come in handy at times, until you get the hang of it.  On a related note, Duncan hits us up with some information on useful add-ons for Storage VMotion.
• Virtual Machine High Availability:  Still listed as an “experimental” feature in VI3 version 3.5, if I recall correctly, Virtual Machine HA uses heartbeats from the VMware Tools inside a guest to try to determine if a guest has failed.  Anyone out there doing more than just experimenting with this?
• Delete all snapshots:  For those end users that don’t work with snapshots, this article is a must read.
• VMotion Is Disabled After ESX Server Upgrade:  This can be handy if you were wondering why VMotion suddenly stopped working after the upgrade to ESX Server 3.5.
• Migration will cause the virtual machine’s configuration to be modified:  It’s still not clear exactly why VirtualCenter is making some changes to virtual machines during a live migration.  Duncan’s explanation about virtualized MMU and paravirtualization support in ESX Server 3.5 makes sense, but what about the commenter’s issue with a migration from ESX Server 3.0.1 to ESX Server 3.0.2?  That doesn’t seem to make any sense, especially on identical hardware.

Anyone with additional information on any of these topics is invited to speak up in the comments.

An issue has been discovered that can cause a CPU spike when VMotion is used in a DRS-enabled cluster with ESX Server 3.5 and VirtualCenter 2.5.  It is my understanding that this can occur with both DRS-initiated VMotion operations as well as manually-initiated VMotion operations.

Fortunately, there’s a workaround for this issue which involves editing the vpxd.cfg on the VirtualCenter server.  Full details on the change that needs to be made, as well as on the log entries that you should see on ESX Server afterward, are found in this article.

UPDATE:  It appears that VMwarewolf has had to pull the original article that described this problem and the workaround.  Fortunately, Google has a long memory, and here’s the workaround for this problem.

Immediately after the “<vpxd>” line in vpxd.cfg, add the following lines:

<cluster>
<VMOverheadGrowthLimit>5</VMOverheadGrowthLimit>
</cluster>

I’m guessing that this information may not be information that VMware wants easily disseminated to the world, or that the workaround has not been fully and completely tested.  So, use this information at your own risk.

In the meantime, this Google search will return the now-unavailable page; use the Cached link to see the workaround and the details from the log file that will help troubleshoot the problem.

UPDATE 2:  VMware has now published this KB article about the issue, along with the workaround for the problem.