Ravello Public Highly Available Load Balancer Failover Configuration

README.md

Preface

This gist provides a way to automatically move an associated Elastic IP between nodes of a cluster in case of failover. This is because the standard VIP infrastructure provided by VRRP is not necessarily transparently compatible with Ravello's public IP/NAT routing, so the private VIP is not routable.

Setup

1. As explained in Ravello's Blog, configure a couple of Highly Available Load Balancers using Keepalived (check this blog article if in doubt).
2. Make sure to enable cloud-init on the VM and configure your VM to export the environment variables described below according to your setup.
3. Make sure you have python installed and install the ravello-take-elastic-ip.py script in such a way the keepalived daemon can execute it.
4. Configure your keepalived.conf to invoke the ravello-take-elastic-ip.py when the node takes over as MASTER.

Manual test

Executing the following line in one of the nodes by hand should allow you to switch the Elastic IP between the nodes:

RAVELLO_CREDENTIALS="<user@domain.com>:<password>" \
RAVELLO_EVIP="<ip-address>" \
RAVELLO_VM_JSON=/etc/ravello/vm.json \
./ravello-take-elastic-ip.py

You shoudl see output similar to the following:

VM FQDNS: <node-host-name>.srv.ravcloud.com
Querying application: 56131616 for current holder for EVIP: <ip-address>
- This VM: 	<this-vm-name>
- Master VM: 	<vm-that-currently-holds-the-elastic-ip>
*** Updating application to switch Elastic Virtual IP from "<original-vm-name>" to "<this-vm-name>" ***
- Done

keepalived.conf

vrrp_instance MyVRRPInstance {
 [...]
 notify_master ravello-take-elastic-ip.py
}

ravello-take-elastic-ip.py

#!/usr/bin/python
import json
import os
import sys
from ravello_sdk import *

env_credentials = os.getenv('RAVELLO_CREDENTIALS', '')
env_evip = os.getenv('RAVELLO_EVIP', '')
env_vm_json = os.getenv('RAVELLO_VM_JSON', '')

if (not env_credentials or not env_evip or not env_vm_json):
  sys.exit(
    """Make sure the following environment variables are set:
    RAVELLO_CREDENTIALS="{0}"
    RAVELLO_EVIP="{1}"
    RAVELLO_VM_JSON="{2}"
    """.format(env_credentials, env_evip, env_vm_json)
  )

credentials = env_credentials.split(':', 2)
EVIP = env_evip
vm_json = env_vm_json

with open(vm_json) as data_file:    
    vm_data = json.load(data_file)

app_id = vm_data['appId'];
vm_fqdns = vm_data['fqdns'][0]

print('VM FQDNS: {0}'.format(vm_fqdns))
print('Querying application: {0} for current holder for EVIP: {1}'.format(app_id, EVIP))

client = RavelloClient()
client.login(credentials[0], credentials[1])

def get_app():
  application = client.get_application(app_id)
  vms = application['design']['vms']
  
  this_vm = { 'name': None }
  this_vm_ip_config = {}
  
  master_vm = { 'name': None }
  master_vm_ip_config = {}
  
  for vm in vms:
    for network_connection in vm['networkConnections']:
      ip_config = network_connection['ipConfig']
      
      if (ip_config.get('fqdn', '') == vm_fqdns):
        this_vm = vm
        this_vm_ip_config = ip_config
  
      if (ip_config['needElasticIp'] and ip_config['elasticIpAddress'] == EVIP):
        master_vm = vm
        master_vm_ip_config = ip_config
        
  return { 'application': application, 
           'vms': {
             'this':   { 'name': this_vm['name'],   'ip_config': this_vm_ip_config }, 
             'master': { 'name': master_vm['name'], 'ip_config': master_vm_ip_config },
           }
         }

app = get_app()

if not app['vms']['this']['name'] == app['vms']['master']['name']:
  print('*** Updating application to switch Elastic Virtual IP from "{0}" to "{1}" ***'.format(app['vms']['master']['name'], app['vms']['this']['name']))

  app['vms']['master']['ip_config'].pop('elasticIpAddress', None)
  app['vms']['master']['ip_config']['needElasticIp'] = False

  client.update_application(app['application'])
  client.publish_application_updates(app['application'])

  app = get_app()
  
  app['vms']['this']['ip_config']['elasticIpAddress'] = EVIP
  app['vms']['this']['ip_config']['needElasticIp'] = True

  client.update_application(app['application'])
  client.publish_application_updates(app['application'])

  print('- Done')
else:
  print('*** Will not update app, this VM is already associated with the Elastic Virtual IP ***')