Skip to content

Instantly share code, notes, and snippets.

@vagelim
Last active July 30, 2020 15:16
Show Gist options
  • Save vagelim/64b355b65378ecba15b0 to your computer and use it in GitHub Desktop.
Save vagelim/64b355b65378ecba15b0 to your computer and use it in GitHub Desktop.
Event notification listener for OpenStack
########################################
# Required changes to nova.conf #
########################################
# notification_driver=nova.openstack.common.notifier.rpc_notifier
# notification_topics = notifications
# notify_on_state_change = vm_and_task_state
# instance_usage_audit_period = hour
# instance_usage_audit = True
# default_notification_level = INFO
# notify_api_faults = true
###### Restart Nova services after making the above changes #######
# Environment Variables: $AMQP_USER, $AMQP_PASSWORD, $AMQP_HOST
import os
from kombu import Connection, Exchange, Queue
from pprint import pprint
nova_x = Exchange('nova', type='topic', durable=False)
info_q = Queue('notifications.info', exchange=nova_x, durable=False,
routing_key='notifications.info')
def process_msg(body, message):
print '='*80
pprint(body)
message.ack()
# Maybe wrap these in try/except: will raise KeyError if not found
AMQP_USER = os.environ['AMQP_USER']
AMQP_PASSWORD = os.environ['AMQP_PASSWORD']
AMQP_HOST = os.environ['AMQP_HOST']
# Maybe wrap this in a try/except: will raise socket.error if connection refused
with Connection('amqp://{0}:{1}@{2}//'.format(AMQP_USER, AMQP_PASSWORD, AMQP_HOST)) as conn:
with conn.Consumer(info_q, callbacks=[process_msg]):
try:
while True:
conn.drain_events()
except KeyboardInterrupt:
exit()
@talwai
Copy link

talwai commented Dec 8, 2015

Also the shebang means that users must install kombu globally, which they may not want to do if they care about their environment. It depends on the context of the article i suppose - how would you prescribe that people use this script?

@talwai
Copy link

talwai commented Dec 8, 2015

Will this queue have the same name across OpenStack versions? https://gist.github.com/vagelim/64b355b65378ecba15b0#file-openstackevents-py-L19

DevStack has kilo i believe, and corgi@datad0g.com has Juno. let me know if you'd like me to test out the latter

@talwai
Copy link

talwai commented Dec 8, 2015

Can you post a brief sample of what the output would look like?

@vagelim
Copy link
Author

vagelim commented Dec 8, 2015

Each question, in turn:
1: yes, that is a good idea
2: Nova uses Kombu to connect to the RabbitMQ environment [0] so it should be available in the global namespace by default. I did not have to install anything to get it working locally for me
3: This queue was defined in the Notification spec ~2011 so it should be consistent across versions, but testing is welcome!

4: (truncated) output appears below

{ 
[...]

 'event_type': 'compute.instance.delete.start',
 'message_id': '45731b78-ceba-48a4-b80c-3ef971dd632d',
 'payload': {             
'image_meta': {             
'progress': u'',
            'ramdisk_id': 'd768f34d-bb21-4afa-97ed-8d7143a43751',
            'reservation_id': 'r-gz5vu80w',
            'root_gb': 1,
            'state': 'active',
            'state_description': 'deleting',
            'tenant_id': '3fb0de5c53434c54829b7150129dec61',
            'terminated_at': u'',
            'user_id': 'adf56761cda54d4c99de59dc50fd6c06',
            'vcpus': 1},
 'priority': 'INFO',
 'publisher_id': 'compute.devstack',
 'timestamp': '2015-11-24 18:22:51.665797'}

================================================================================

@vagelim
Copy link
Author

vagelim commented Dec 8, 2015

I also made a slightly modified version of the script in this gist which is much less verbose, which I used to correlate event start/end times:

#!/usr/bin/env python
########################################
#    Required changes to nova.conf     #
########################################
# notification_driver=nova.openstack.common.notifier.rpc_notifier
# notification_topics = notifications
# notify_on_state_change = vm_and_task_state
# instance_usage_audit_period = hour
# instance_usage_audit = True
# default_notification_level = INFO
# notify_api_faults = true
###### Make sure you restart Nova#######

from kombu import Connection, Exchange, Queue
from pprint import pprint

nova_x = Exchange('nova', type='topic', durable=False)
info_q = Queue('notifications.info', exchange=nova_x, durable=False,
               routing_key='notifications.info')

def process_msg(body, message):
    print '='*80
    print body['event_type']
    print body['publisher_id']
    print body['timestamp']
    #pprint(body)
    message.ack()

#Change guest:guest and the hostname to match your configuration
with Connection('amqp://guest:guest@localhost//') as conn:
    with conn.Consumer(info_q, callbacks=[process_msg]):
        try:
            while True:
                conn.drain_events()
        except KeyboardInterrupt:
            exit()

which produces output like:

compute.instance.exists
compute.devstack
2015-12-08 19:00:12.966361
================================================================================
floatingip.update.start
network.devstack
2015-12-08 19:35:23.684770
================================================================================

@talwai
Copy link

talwai commented Dec 8, 2015

Well 2. is not an assumption you can make about all hosts running Nova. People will install nova and dependencies into virtual environments and wherever else they like if they don't want to interfere with their system python. I would suggest you drop the shebang and suggest people run it like </path/to/nova/env/>bin/python openstack_events.py. You can probably trust people to know where this directory is. If not, then maybe: wrap this in a PyPi package that ships kombu as a separate dependency OR write a bash script that creates a virtualenv, installs kombu, runs venv/bin/python openstackEvents.py

@vagelim
Copy link
Author

vagelim commented Dec 9, 2015

Made the changes to environment variables.
I think the bash script option is best. I quickly wrote one here: https://gist.github.com/vagelim/98c8792ba12dc8f90341
@talwai

@gpion
Copy link

gpion commented Apr 30, 2018

Hello,
I had to replace routing_key='notifications.info' by routing_key='notifications.*' since I was not catching some nova messages like instance.create.{start,end} amongst others.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment