Skip to content

Instantly share code, notes, and snippets.

What would you like to do?
TripleO fencing config for fence_xvm
FENCE_XVM_KEY=${FENCE_XVM_KEY:-$(cat /etc/cluster/fence_xvm.key)}
MULTICAST_ADDRESS=${MULTICAST_ADDRESS:-$(grep address /etc/fence_virt.conf | head -n1 | awk -F'"' '{ print $2}')}
if [ -z "$FENCE_XVM_KEY" ]; then
echo 'ERROR: fence_xvm key not set' 1>&2
echo '$FENCE_XVM_KEY is empty and /etc/cluster/fence_xvm.key does not exist / cannot be read / is empty' 1>&2
exit 1
if [ -z "$MULTICAST_ADDRESS" ]; then
echo 'ERROR: multicast address not set' 1>&2
echo '$MULTICAST_ADDRESS is empty and trying to read it from /etc/fence_virt.conf did not work' 1>&2
exit 1
MACHINES=$(virsh list --all | grep "$MACHINE_REGEX" | awk '{print $2}')
echo "{ \"devices\": ["
MACHINE_MAC=$(virsh dumpxml $MACHINE | grep 'mac address' | awk -F"'" '{print $2}')
echo " {"
echo " \"agent\": \"fence_xvm\","
echo " \"host_mac\": \"$MACHINE_MAC\","
echo " \"params\": {"
echo " \"multicast_address\": \"$MULTICAST_ADDRESS\","
echo " \"port\": \"$MACHINE\","
echo " \"manage_fw\": true,"
echo " \"manage_key_file\": true,"
echo " \"key_file\": \"/etc/fence_xvm.key\","
echo " \"key_file_password\": \"$FENCE_XVM_KEY\""
echo " }"
echo " }$([ \"$MACHINE_COUNT\" = \"$MACHINE_NUM\" ] || echo -n ',')"
echo "]}"

TripleO virtualized deployment -- fence_xvm config

  1. Prepare for fence_xvm multicast traffic

By default, overcloud has no direct connection to host and multicast traffic will not pass through undercloud, which prevents fence_xvm from working. This needs to be worked around somehow:

Option A: connecting the host machine directly to br-ctlplane

  • Instead of talking to the undercloud and overcloud through libvirt's default network, we'll talk to br-ctlplane directly via brbm on the host machine. This will drop your connections to undercloud and overcloud and you'll need to re-establish them. Set up the routing on the host machine:
ip addr add dev brbm
ip link set brbm up
# ^ this will automatically set up a route like
# dev brbm  proto kernel  scope link  src

# now you need to delete the original route through default libvirt network
ip route del via dev virbr0
# ATTENTION: you'll have a different IP here ^

Option B: multicast forwarding on undercloud

  • On undercloud, install smcroute for forwarding multicast traffic between overcloud and host
xz -d smcroute-2.0.0.tar.xz
tar -xf smcroute-2.0.0.tar
cd smcroute-2.0.0
  • Configure and run smcroute in foreground (e.g. in a separate tmux/screen window)
echo '
mgroup from eth0 group
mroute from eth0 group to br-ctlplane

mgroup from br-ctlplane group
mroute from br-ctlplane group to eth0
' > smcroute.conf
./smcroute -d -n -f smcroute.conf
  1. Configure and run fence_virtd on baremetal

  • Install and configure fence_virtd:
yum -y install fence-virtd-libvirt fence-virtd-multicast
fence_virtd -c
# use the defaults
# multicast address, port 1229
# ATTENTION when selecting network interface:
# * if you connected baremetal to br-ctlplane (option A), use brbm
# * if you went with multicast forwarding previously (option B), use virbr0
  • Configure fence_xvm secret:
mkdir /etc/cluster
echo -n "abcdef" > /etc/cluster/fence_xvm.key
  • Run fence_virtd in debug mode in foreground to see what it does (e.g. in a separate tmux/screen window)
fence_virtd -F -d99
  1. Add fencing parameters to your overcloud stack

  • Add fencing parameters to your custom heat environment file ($OVERCLOUD_CUSTOM_HEAT_ENV). Use the script to create the JSON value for FencingConfig parameter.
  EnableFencing: true
  FencingConfig: ##### here belongs output of #####

Now you can deploy overcloud and it should be configured with fence_xvm devices for all controllers and monitoring should report that the machines are started.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.