ESXi

This is TA2659, Managing ESX in a COS-less World. The focus here is on tools that allow you to manage ESX without using the Service Console.

When I arrived at the session, the presenter was discussing VMware’s goal to drive parity between the “actual” CLI present in the Service Console and the Remote CLI. There is a desire to ensure that any command that can be run on ESX or ESXi can also be run on the other.

Another tool to use in managing ESX without the Service Console is the VI Toolkit for PowerShell. I’m sure that most readers are already familiar with the VI Toolkit, so I won’t go into any great detail there. There are about 120 cmdlets or so in the Toolkit today; another 50 or so are slated for release in the next version of the VI Toolkit. In addition, VI Toolkit management/compatibility is a core design facet—everything needs to be manageable via the VI Toolkit.

The VI Perl Toolkit is another scripting toolkit that can be used to manage ESX without relying upon the Service Console. This works on both Linux and Windows, where as the VI Toolkit only works on Windows. The vicfg-* tools are built on Perl.

Future directions include enhancements to the VI Perl Toolkit to expose functionality as Perl functions. A VI Java Toolkit will be available soon (within a month?). Other toolkits may become available depending upon market demand and direction.

From the perspective of server health monitoring, CIM SMASH is the direction in which VMware is moving. CIM (Common Information Model) is both a protocol and a data model representing functionality. SMASH (Systems Management Architecture for Server Hardware) is the data model for hardware health monitoring and management. An example of a tool that works with CIM SMASH is WinRM, which ships on Windows Vista and presumably Windows Server 2008. CIM SMASH profiles simply define the sensors and the values that should be retrieved for various hardware elements.

A fair amount of CIM SMASH functionality was exposed in ESX 3.5 Update 2. This is done per-host; in the future, VirtualCenter will aggregate that for multiple hosts.

An attendee asked a question about varying degrees of hardware monitoring being exposed in the initial release of ESXi and CIM support within both ESXi and the underlying hardware. The response is that this functionality is driven by both SIM providers, which are written by either VMware or the hardware vendor; in most cases, it’s VMware ESXi that needs to beef up the SIM providers.

In the future, the vision of VMware is to abstract the configuration into a configuration file. This configuration template could then be applied to multiple hosts, and the configuration can be assessed regularly to verify configuration, compliance, etc. The information required to do all this is already being handled by the web services API in VirtualCenter, and the tools necessary to perform the configuration are already present in the API (the esxcfg-* and vicfg-* commands already leverage the API to do these very tasks). Combining these two things would allow us to create a configuration template that can be applied to a host while still allowing for customization (like VM customization during cloning).

The subject of deployment is a key issue when we think about losing the Service Console. One approach to handling these issues is deploying physical machines; another would be to deploy virtual machines to handle these tasks. Partners could wrap up the agents that would typically be deployed in the Service Console as a virtual appliance, but then users could end up with numerous virtual appliances. What if VMware were to provide a virtual infrastructure management appliance? That’s what VIMA (Virtual Infrastructure Management Assistant) is.

VIMA is a virtual appliance packaged as OVF and is distributed, maintained, and supported by VMware. This is downloaded and installed by the customer according to their management procedures. This will be a well-known deployment environment that partners can rely upon being present. This will be a 64-bit Linux distribution with VMware Tools, VI Perl Toolkit, the Remote CLI (now known as the VI CLI), and a JRE already present. VIMA can be patched for updates, and it allows you to manage one or more VMware ESX hosts directly or through VirtualCenter. VIMA can enable agents to authenticate themselves, and VIMA will rotate its passwords on the hosts. Additionally, sample code and documentation will be available for programming applications to work in VIMA.

In “classic” ESX, management agents and hardware agents ran in the Service Console; with VIMA, updated management agents will talk through the VI API and hardware agents will talk through CIM SMASH. An example of this is the APC PowerChute Network Shutdown (PCNS), which is being rewritten to use the VI API and will run in VIMA.

Anyone interested in VIMA can e-mail vima_request@vmware.com and request access to pre-GA versions of VIMA. VIMA is expected for general release in the fourth quarter of this year. All VIMA releases will work with both ESX and ESXi (again, pointing to the desire to keep parity between these two products).

Future versions of VIMA may add Active Directory support; authentication through vCenter Servers; improved automation, configuration, and updates; UI integration in the VI Client; additional VMware components pre-installed. Finally, VMware Studio will be used to build future versions of VIMA.

At this point, the presentation ended and the floor was opened up for a question and answer session.

Once again, there’s no wireless signal in the breakout session, so I’ll have to publish this in a delayed fashion. Ugh! Someone needs to work on this wireless network.

This is session TA2668, VMware ESX Architectural Directions. This is another session that provides forward-looking information at future versions of VMware ESX.

The presenter started out the presentation by giving an overview of VMware ESX, what it’s designed to do, what features it has currently, what it’s capable of doing, what it knows, how it manages CPU scheduling and other resources like memory. This would include stuff like being aware of the difference between sockets, cores, and hyperthreads; and stuff like VMware ESX’s advanced memory management functionality (like transparent, content-based page sharing). He also reviewed the management of I/O such as network and storage I/O.

Moving into “future features,” these features will focus on security, scalability, interoperability, and performance.

In the security area, VMware is focusing on improving security in a number of areas. One of these is reducing the size of the platform, such as in ESXi; it is anticipated that this will reduce the surface area and lower the number of potential vulnerabilities. VMware is also looking at the use of ASLR (Address Space Layout Randomization) and NX support (No eXecute). Together these features will protect both applications and the kernel and make exploits much more difficult and more difficult to automate and repeat.

In addition, VMware is working on the VMsafe APIs, previously announced at VMworld Europe 2008 in February. These APIs will allow VM introspection and intervention. Security modules will run outside the guest and are strongly isolated from the guest(s) they are monitoring. However, these modules will still have full access, full visibility, and full introspection and control functionality.

On the scalability front, VMkernel will move to 64-bit. Although in practice this will be unnoticeable, it will simplify VMkernel on large memory machines and on systems with larger numbers of CPUs. This will not impact support for 32-bit and 64-bit guests running on VMware ESX. This change will enable support for up to 64 logical processors, up to 512GB of RAM, up to 512 vCPUs per host, and higher VM limits (8 vCPUs and 256GB of RAM per VM). These higher limits will also help improve support for guests with 2 and 4 vCPUs.

VMware ESX will also leverage dynamic frequency and voltage scaling, which will enable the VMkernel to manage power states for CPUs when load decreases. This will be accomplished via Intel Enhanced SpeedStep and AMD PowerNow.

Power will also be treated as a first-class resource in the VI Client, with tracking and reporting of power utilization.

Another scalability improvement is the Distributed vSwitch (DVS). The DVS is a vSwitch that spans an entire VMware ESX cluster. This brings greatly simplified network configuration across the entire cluster, stateful vSwitches (vSwitches can maintain per-port policies), and plugins. Examples of plugins would include appliance APIs (to create inline filters for per-VM traffic) or switch APIs (to modify the forwarding algorithm).

vStorage Thin Provisioning is another scalability improvement. However, as mentioned in my earlier post on BC2621, this may conflict with VMware FT. This is intended to reduce disk space utilization, improve disk-related operations like backup, cloning, etc. Obviously, new alerts need to be created to manage disk allocation and overprovisioning.

Moving ahead to interoperability, the presenter first discussed Enhanced VMotion Compatibility (EVC). The idea behind EVC is masking certain CPU features so that guests can migrate live via VMotion between hosts with dissimilar CPUs. EVC leverages functionality built into modern processors from AMD and Intel CPUs to hide CPU features so that CPUs appear to be identical to guests. EVC is available today in VMware ESX 3.5 Update 2. EVC problems are detected when a host is added to a cluster, to prevent problems before a user attempts a VMotion between hosts with incompatible CPUs.

The Service Console will be updated to a 64-bit distribution running on version 2.6 of the Linux kernel. All hardware device drivers will be in VMkernel; none of them will be in the COS. In fact, the COS filesystem will reside in a VMDK on a VMFS and uses the same storage path(s) as VMkernel. Of course, there is no Service Console for ESXi. Both ESX and ESXi support CIM-based host management.

Another area of interoperability is with storage plugins, using the VMware Pluggable Storage Architecture (PSA). This will enable partners to write plugins to enhance storage functionality. VMware ESX will ship with a plugin known as NMP (Native Multipathing Plugin), which in turn is comprised of SATP (Storage Array Type Plugin) and PSP (Path Selection Plugin). ALUA support will be added as well.

VMDirectPath I/O is another interoperability advancement. This allows the guest to directly control physical hardware. This seems to fly directly in the face of virtualization, which is intended to virtualize and abstract physical hardware away from the virtual machines. However, this could be useful in some instances. Hardware I/O memory management is required in order to isolation guest memory access and translate guest addresses to host addresses. In addition, PCI device reset capability is required.

VMDirectPath I/O does introduce its own set of challenges, such as the impact on VM suspend, checkpoint, and migration. VM memory management is also impacted. It is anticipated that the “Gen1″ release of this functionality will accept these limitations and “Gen2″ will begin to address them.

VMware will also support newer device drivers from partners and will also allow asynchronous device driver updates. A device driver development kit will be available to allow 3rd party developers to add device drivers to VMware ESX. Of course, with the switch to a 64-bit architecture, the drivers will also switch to 64-bit drivers.

Finally, in the area of performance, VMware will improve CPU efficiency for I/O virtualization. This covers an enormous amount of work for reducing overhead, enabling large packet send/receive, scheduling processor interrupts, and the paravirtualized SCSI device to improve SCSI performance. Future versions of ESX are also expected to include large page support. Other areas of improvement include expanded support for hardware virtualization, and VMware will focus on specific application areas to help drive performance and optimization of those applications on VMware ESX. This is in addition to helping customers optimize VMware ESX itself.

At this point, the presenter concluded the session.

Now that I’ve gotten through the first part of discussing VMware’s VDC-OS announcement—making sure that the message and vision is a bit clearer—I can focus on some of the specifics contained within the announcement. In other words, I can talk about new features. (That should make Duncan happier.)

• VMware Fault Tolerance (FT), formerly “Continuous Availability,” (demoed at VMworld 2007, described in my liveblog here) provides real-time VM mirroring between two different hosts. If a host fails, the mirrored VM on the secondary host picks up automatically with no disruption to the users. Marathon Technologies is working on similar functionality for Citrix XenServer, and both companies are expected to deliver next year. If VMware wants a leg up on the competition, they need to deliver this first.
• The Distributed vSwitch enables administrators to define network settings at the cluster level, instead of on a host-by-host basis. This is huge for larger shops. Define your port group once, and you’re done.
• As has been pointed out elsewhere, Cisco is announcing the first third-party vSwitch for VMware ESX. Alessandro points out rumors that it will run NX-OS and will be a distributed vSwitch. In any case, it will certainly bring hard-core Cisco shops closer to embracing VMware as it will give them end-to-end control over the networking environment, all the way down to the virtual port level within any given VMware ESX host.
• vStorage Thin Provisioning will help on storage demands. I suspect this is just the use of thin provisioned VMDKs, but if anyone has any other information to share I’d love to hear it.
• Similarly, vStorage Linked Clones just brings to VMware ESX a feature that VMware Workstation and VMware Lab Manager have had for a while.
• VMware will finally enter the backup market with vCenter Data Recovery, which (to my understanding) will leverage VCB to provide a backup solution for the virtual infrastructure.

That’s quite an impressive list of features slated to be included in VDC-OS. What I’m also interested in seeing, though, is how the underlying components of VDC-OS—VMware ESX and VirtualCenter/vCenter—are going to change. I’ve seen rumors of 64-bit VMkernel and Console OS, and Duncan himself points out linked vCenters (which is quite cool, might I add). I suppose that will be part of the “Tech Preview” sessions that are going on later this week.

OK, that’s it for now; need to run off for another session. Stay tuned here and on Twitter for more information!

So, the news is out on VMware’s new announcement regarding the Virtual Datacenter OS (VDC-OS). You can find more coverage of this on a few other sites:

Virtual Datacenter OS from VMware
VMware moving to cloud-computing with Virtual Datacenter Operating System (VDC-OS)
VMware testing data center OS for managing, literally, everything
VMware Tries to Expand Throughout the Data Center

I’m kind of excited to be able to talk about this, because it was a topic of the Partner Technical Advisory Board today and I didn’t think I’d be able to talk about it until later in the week.

First off, let me just state that the Computerworld article linked above (”VMware testing data center OS for managing, literally, everything”) is just plain wrong. What VMware is working on is an OS for the virtual datacenter, not a virtual OS for the datacenter. The distinction is important. VDC-OS isn’t intended to be an OS for all aspects of the datacenter. It’s intended to be an OS for all aspects of the virtual datacenter.

When you think of an OS, you think of software that manages access to resources and provides services to applications. That’s what VMware is doing with VDC-OS: managing access to resources and providing services to applications, only this time the applications are workloads (virtual machines with an OS and a set of applications on that guest OS). VDC-OS will provide sets of services to these applications:

• Application vServices, like availability, security, and scalability. These application vServices are provided via features like VMotion, Storage VMotion, VMware HA, VCB, and—in the future—stuff like VMware Fault Tolerance (FT), formerly known as Continuous Availability. See this page on VMware’s site for more examples.
• Infrastructure vServices, like compute functionality (vCompute), networking connectivity (vNetwork), or storage features (vStorage). These vServices are manifested as features like VMware DRS, and will be extended in the future with things like vStorage Linked Clones, or 3rd party virtual switches, or VMDirectPath. The APIs are there for additional third party vServices to be added as well; one example would be network load balancing as an infrastructure vService.
• Cloud vServices enable the interaction of on-premise infrastructure (the servers in your data center) to integrate with external cloud infrastructure. There are no concrete examples to really share here; in my opinion, this is the most nebulous part of this announcement. See this page for more information.
• Finally, management vServices provide…well, management functionality for the virtual data center and the applications running in the virtual data center. More information is available here.

The way to really view VDC-OS is not as a “datacenter OS”, because it’s not intended to provide automation of non-virtual resources. Instead, look at VDC-OS as a framework. Within this framework are sets of services that can be extended or modified in very standardized ways (via APIs and SDKs) to provide different functionality for the applications running within that framework. Some third party ISV wants to write a different way of providing fault tolerance and failover? Fine, no problem, that can be plugged into the VDC-OS application vService framework for availability. I used the example earlier of a load balancer as an infrastructure vService. VMware announced the VMsafe APIs back at VMworld Europe, and the idea of VMsafe ties directly into API access for ISVs to develop new or different security-related application vServices that can be provided to all applications running within the VDC-OS, such as anti-virus or host-based intrustion detection/prevention.

I was a bit worried that this messaging wasn’t going to be received as clearly as I had hoped it would be, and the initial coverage I’m seeing so far confirms that many people are going to misread what VMware is trying to do. Hopefully, we can get the idea across to everyone so that they can begin to really understand where VMware is headed with this idea and why it is the next step in the evolution and maturation of virtualization.

I guess I’m a little behind the times here, but a little company named Bluebear is in the process of developing a cutting-edge cross-hypervisor management tool named Kodiak.

Kodiak is in private beta right now, so we can’t test it ourselves to see what it looks like. Based on the screenshots available on the website, and through what I’ve seen in talking with one of the developers in the #vmware channel on IRC, it’s pretty awesome. And the best feature? It’s cross-platform! Written in Adobe AIR, it runs on Windows, Linux, and (my personal favorite) Mac OS X. In addition to being cross-platform, it’s also intended to be hypervisor-neutral, managing VMware, Xen, and Hyper-V. Only VMware support is present right now, but support for the others is coming.

If Kodiak ends up being as good as it looks like it will be, this will be awesome tool. As a VM admin, you’ll want to stay on top of the development of this tool.

News, views, events, and commentary from around the virtualization world that struck my fancy over the past few weeks—that’s what you’ll find in Virtualization Short Take #17!

• I stumbled across this series of podcasts on building a scalable virtual desktop deployment. It’s a four part series: Part 1, Part 2, Part 3, and Part 4. I’m still struggling with finding a way to incorporate podcasts into my day; I’m drinking from the firehose as it is. For those that do listen to podcasts, perhaps this series will prove helpful.
• Apparently, the argument about VMware violating the GPL (I wrote about that here quite some time ago) has surfaced again. Gordon Haff promptly and rather cleanly squashes that concern. Well said, Gordon.
• Tim Jacobs posts a great article on matching LUNs between ESX and the VCB proxy server. This is just one of several good posts by Tim; here’s another one on VSS snapshots with ESX 3.5 Update 2. This is a web site to watch.
• Ben Armstrong reminds us that Hyper-V requires NTFS in order to work. Isn’t it time for FAT32 to go away?
• Will VMware ESX/ESXi 3.5 Update 2 be the first hypervisor validated under the SVVP? Gabe posts some information on his site to that effect, indicating that VMware expects to announce validation before VMworld. That would be a strong competitive advantage, indeed.
• VMware is also hitting hard with performance advantages and ties to hardware acceleration, as indicated by John Troyer’s post regarding VMDq, VMware Netqueue, and VMDirectPath (seen via VMblog.com). The coolest part about this stuff is that a lot of it is already available. We’re not waiting for a future version of VMware Infrastructure to take advantage of this stuff; instead, we’re waiting for the hardware. Now how’s that for a change!
• Rich over at VM /ETC gives us a great breakdown on licensing Windows VMs for a non-Microsoft virtualization solution. He comes to the same conclusion some of my customers have already made: licensing Windows Server 2008 Datacenter Edition may make the most sense. Good job, Rich!
• Duncan alerts us to a potential problem with HP Insight Manager agents on VMware ESX 3.5 Update 2. I share Duncan’s view; I try to avoid agents in the Service Console wherever possible. That’s why I was glad to see VMware add the Health Status functionality in Update 2, as this gives you some hardware monitoring functionality without any agents in the Service Console. I don’t know that I’ve quite arrived at the same place as Duncan; I like the Service Console. I guess that’s because I’m accustomed to it. In any case, if you’re running HP hardware with the IM agents, be on the lookout for this issue. Thanks for bringing this to everyone’s attention, Duncan!
• Interested in keeping up with all the various goings-on at VMworld? David Davis turns us on to a few additional sources of information (as if we needed anything more!).

Also, I’m excited to announce that my blog has been selected for inclusion in a new blog aggregation site, VirtualizationFeed.com. Patrick over at Microsoft announced the new site yesterday via the Virtualization Team Blog with this post. I’m thrilled to be included and I hope to be able to continue to deliver quality information. Thanks for reading!

Altor Networks is one of several companies that has emerged from stealth mode over the last six months or so to pitch their products for virtualization security. Altor Networks’ product, the Virtual Network Security Analyzer, or VNSA, is their flagship product, but it’s important to understand what the VNSA is and what it attempts to do. This product isn’t intended to provide network security through enforcement or policy control, but instead is intended to enhance network security through visibility.

When organizations move large numbers of their servers into a virtualized environment, they often lose network visibility in that virtualized environment. This can, in theory, create a security problem because traffic between VMs can’t be monitored. One compromised VM could lead to another compromised VM, etc., because the traffic isn’t visible to existing security solutions.

<aside>Now, not being a security expert, I have to ask myself: is such a scenario really that likely? It seems to my uneducated mind far too likely that some sort of traffic would “leak” out onto the physical network and be detected there. Or am I just completely misinformed? Anyway, I digress…</aside>

The VNSA has a two-tier architecture:

• Each ESX server hosting VMs whose traffic should be monitored must have an Altor Agent VM, a small virtual appliance that can monitor up to three separate vSwitches. (This limitation, by the way, is an ESX limitation; VMs are limited to four virtual NICs. Three NICs can be used for monitoring, and the fourth NIC is used for reporting and management.) The Altor Agent relies upon the use of a dedicated port group on each vSwitch that allows promiscuous mode and uses VLAN ID 4095. VLAN ID 4095 is the “special” number that tells ESX to pass traffic from all VLANs up to the guest VM.
• The various Altor Agents report back to a central VM, known as the Altor Center. This is another virtual appliance that is designed to accept the information recorded and reported by the agents. Altor Center also integrates with VMware VirtualCenter to retrieve the list of virtual machines available on the various hosts.

Altor Center provides a web-based interface to view the information gathered and reported by the agents. There are a variety of different reports available, and Altor Center can show traffic patterns between groups of VMs. Users can drill down to see traffic patterns from a variety of perspectives.

While VNSA does a good job of providing network visibility, it does not provide network traffic enforcement functionality. You can’t knock the VNSA for failing to enforce security policies considering that it wasn’t intended to do so. Altor has publicly stated that future products will build upon the foundation established in the VNSA and those future products will provide security policy enforcement. This distinction is important because otherwise users may be inclined to compare virtual security solutions with different feature sets and different target purposes. Comparing the VNSA with a product that is essentially a firewall is not a valid comparison. As a user, if you are looking for a solution to provide visibility into the virtual network environment, VNSA is one solution. If you are seeking security policy enforcement, however, you will need to look elsewhere. VNSA was not intended to fill that need.

Much has transpired since yesterday, when I urged VMware to join the SVVP and get their software validated for full support by Microsoft. Since that time, it has come to light that VMware has joined the SVVP, although a formal announcement has not yet been made, and Microsoft has announced some significant licensing changes regarding virtualization. I’ve been reading the various announcements and analyses regarding this information and I thought it might be beneficial to try to pull all this together.

First, refer to Patrick O’Rourke’s blog entry, which does a great job of summarizing the need for application mobility licensing. Clearly, customers needed the ability to move applications freely between physical servers, and Microsoft themselves needed to allow customers to do this now that they have a more robust virtualization solution in place (Hyper-V and SCVMM 2008). While the licensing changes do benefit all virtualization vendors, it’s important to note that Microsoft needed these changes for themselves as well.

Patrick’s post also brings to light that while VMware has joined the SVVP, cooperative support is not yet in place. That won’t come until validation via SVVP is completed, which may take some time. The joining of SVVP was necessary, as it is merely one step toward a larger goal.

However, there’s more here than perhaps many people are realizing. Fortunately, there are a number of sites out there pointing out important caveats to the new licensing changes:

• Rich at VM /ETC correctly points out that the new licensing does not apply to the Windows Server OS itself. So you are still going to have problems with VMware HA and VMware DRS automatically moving VMs from server to server unless you use Windows Server Datacenter Edition (see below).
• Chris Wolf points out (both on his personal blog as well as the Data Center Strategies blog) that the lift on the 90-day license transfer does not apply to licenses purchased outside of a volume license agreement. Using OEM licenses? Then you’re out of luck; those licenses still fall under the old restrictions.
• eWeek’s Joe Wilcox points out that because the Windows Server OS isn’t included in the 90-day license relief, some customers will simply license Windows Server Datacenter Edition for every CPU in their data center. Of course, the fact that you now get Hyper-V for free with Windows puts Microsoft…ahem, ahead of the game, shall we say? Read Joe’s full report here.

So, while Microsoft’s licensing changes are a good first step, there’s still more work to be done. Let’s applaud the changes, which were necessary, but let’s continue to press Microsoft to fix the issues that remain.

OK, I’m not a big fan of the whole “LOL cat” kind of thing, but this is really funny!

Reader Rince pointed out this VMware Communities thread that has highlighted a pretty serious problem with VMware Infrastructure 3 version 3.5 Update 2. This problem seems to affect both ESX and ESXi.

The problem is that as soon as the date hits August 12, 2008, all VMs refuse to power on and an error is logged indicating that the product’s license has expired. If VMs are already running, they should be fine; the problem, as I understand, only prevents users from powering on new VMs or performing VMotion operations.

If you are being affected by this bug, then your only real option at this point is to disable NTP (if it is being used) and set your date back a couple of days.

There’s no firm word yet on when a fix will be available from VMware.

Rince, thanks for the heads-up!

Update as of 9AM ET August 12, 2008

Updated ISOs and TARs for Update 2 (let’s call them Update 2 v2) will be available by noon PT tomorrow, August 13. In the meantime, the following workaround should help:

1. Disable NTPd on the ESX hosts.
2. Set the date on the ESX hosts back to some point prior to August 12, 2008. Some people have suggested using the same day and month but a completely different year in the past; this will make it easier to repair ESX log files using search/replace.
3. Set the VM to boot into the BIOS.
4. Boot the VM. In the BIOS, set the date and time properly. This will be incorrect because it gets inherited from the host. This is independent of the time sync functionality in VMware Tools. If you don’t fix this, VMs will boot up with the wrong date and/or time and that will cause additional problems (like Kerberos authentication within Active Directory).
5. After the VM has booted, ensure that time sync within VMware Tools has been disabled. Configure time sync to a reliable source. Active Directory usually handles this for Windows-based systems in a domain.

This workaround is only necessary to power on a VM, resume a suspended VM, or perform a VMotion operation. If all your VMs are currently up and running, just disable DRS.

Some have also suggested disabling HA. If you disable HA, then you will lose the ability for the VMs to be registered on another host if a host in the cluster fails. However, if you leave HA enabled, then without this workaround the VMs won’t boot anyway (but they will be registered). If you choose to leave HA enabled, just be aware that you will need to use the workaround in order for the VMs to actually power up after a host failure.

Hopefully this information will help. I’ll post more information as it becomes available.

Update as of 3PM ET August 12, 2008

VMware will apparently release an express patch to specifically address this issue by 6PM PT (9PM ET) today, August 12, 2008. As soon as I get word that the patch has been released, I will post an update here.

Update as of 11:15PM ET August 12, 2008

An express patch specifically to correct this issue has been released. It is available for download from VMware’s site. Also available there are links to new KB articles for ESX and ESXi that describe deployment and installation considerations.

Update as of 9AM ET August 13, 2008

I received word from VMware that the express patch released last night is fully compatible with VMware Update Manager. This should help simplify the deployment of the patch to affected servers.