Disabling HA will “uninstall” the HA agent on the VMware ESX servers and theoretically clean up any lingering information. Aside from a host failure occuring while HA is disabled, there’s no risk of which I am aware. Even so, prepare appropriately in production environments–just in case.

Killing the “wrong” VMs on each esx would be a solution, but with more than 60 VMs and 6 esx it’s quite a lot of work.

What possibilites do we have to clean up the situation? What will occur if we would disable HA? Will it cleanup the isolation response and no VM will be restarted unexpectedly?

Daniel

I was wondering if this has been logged as a bug with VMware? Clearly if the working hosts are trying to power on the VM before it has been powered off, the software has almost created a split-brain scenario without any mis-configuration - surely a bug?

I’m going to do some testing with ESX 3.5 Update 2 (assuming the timebomb fix is stable) - will post results.

Cheers,
TK

I also create a larger script to help get some additional information from HA.

ha_status.sh

!/bin/sh
# Erik Bussink (July 2008)
#
# requires the FT_DIR variable
# add the following line to you’re /root/.bashrc
# export FT_DIR=/opt/vmware/aam
#

# Monitor AAM Health (Command taken from VMworld 2006 - Effective DRA and HA in production by Nitin Suri)
/usr/bin/perl /opt/vmware/aam/ha/aam_config_util.pl -z -cmd=listnodes -domain=vmware

# List current HA members status
echo
/opt/vmware/aam/bin/ftcli -domain vmware -cmd “la -l”
echo

# List status of AAM Agent on each node
echo
echo -[ esx11 ]———————————————————————
/opt/vmware/aam/bin/ftcli -domain vmware -cmd “status esx11″
echo -[ esx12 ]———————————————————————
/opt/vmware/aam/bin/ftcli -domain vmware -cmd “status esx12″
echo -[ esx13 ]———————————————————————
/opt/vmware/aam/bin/ftcli -domain vmware -cmd “status esx13″
echo -[ esx14 ]———————————————————————
/opt/vmware/aam/bin/ftcli -domain vmware -cmd “status esx14″
echo -[ esx15 ]———————————————————————
/opt/vmware/aam/bin/ftcli -domain vmware -cmd “status esx15″
echo -[ esx16 ]———————————————————————
/opt/vmware/aam/bin/ftcli -domain vmware -cmd “status esx16″

Sure I have not really optimized the bash shell lines near the end, but it’s easy and it can be usefull.

Erik

Good stuff–thanks for sharing it here!

ha_watch.sh

#!/bin/sh
# Created by Erik Bussink (July 2008)
#
# requires the FT_DIR variable
# add the following line to you’re /root/.bashrc
# export FT_DIR=/opt/vmware/aam
#
watch ‘/opt/vmware/aam/bin/ftcli -domain vmware -connect localhost -port 8042 -timeout 60 -cmd listnodes’

#!/bin/sh
# Created by Erik Bussink (July 2008)
#
# requires the FT_DIR variable
# add the following line to you’re /root/.bashrc
# export FT_DIR=/opt/vmware/aam
#

# List current HA members, their Roles and their Status
/opt/vmware/aam/bin/ftcli -domain vmware -connect localhost -port 8042 -timeout 60 -cmd listnodes

echo

# List current HA members settings
/opt/vmware/aam/bin/ftcli -domain vmware -cmd “la -l”

You don’t need to set das.isolationaddress (which defaults to the SC’s default gateway) unless that address does not respond to ping, or unless you’d like to use some other device on the network.

For das.isolationaddress2, set it to be any device that responds to ping, is available via that network adapter, and could be used as a way for the host to determine if it is isolated.

Hope this helps!

Once I create a second SC connection, on a different vswitch on a different subnet I have to: 1) add the das.isolationaddress in Advanced options, and make that the SC gateway. 20 Then add das.isolationaddress2 and - here’s what I don’t get - add what address? The gateway address for that subnet, or the actual address of the second console? Or something else entirely?

Do you have two VMkernel ports? Looking back at your comments, I see mention of a VMkernel port on vSwitch0 as well as on vSwitch1. Unless you are using IP-based storage on the 10.10.10.x network, you don’t need a VMkernel port on vSwitch0, just a Service Console port group. That would leave you with a single VMkernel port on vSwitch1. Or am I misunderstanding your configuration?

Feel free to move this to e-mail if you want. You can get my e-mail address from the About page on this site.