Skip to content

Instantly share code, notes, and snippets.

@brandond
Last active February 6, 2023 21:09
Show Gist options
  • Save brandond/6b4d22eaefbd66895f230f68f27ee586 to your computer and use it in GitHub Desktop.
Save brandond/6b4d22eaefbd66895f230f68f27ee586 to your computer and use it in GitHub Desktop.
Python script to auto-tag AWS EBS Snapshots and Volumes using AMI and Instance tags
import copy
import logging
import os
import boto3
logging.basicConfig(level=os.environ.get('LOG_LEVEL', 'INFO'))
ec2 = boto3.client('ec2')
logger = logging.getLogger(__name__)
def tag_snapshots():
snapshots = {}
for response in ec2.get_paginator('describe_snapshots').paginate(OwnerIds=['self']):
snapshots.update([(snapshot['SnapshotId'], snapshot) for snapshot in response['Snapshots']])
for image in ec2.describe_images(Owners=['self'])['Images']:
tags = boto3_tag_list_to_ansible_dict(image.get('Tags', []))
for device in image['BlockDeviceMappings']:
if 'SnapshotId' in device['Ebs']:
snapshot = snapshots[device['Ebs']['SnapshotId']]
snapshot['Used'] = True
cur_tags = boto3_tag_list_to_ansible_dict(snapshot.get('Tags', []))
new_tags = copy.deepcopy(cur_tags)
new_tags.update(tags)
new_tags['ImageId'] = image['ImageId']
new_tags['Name'] += ' ' + device['DeviceName']
if new_tags != cur_tags:
logger.info('{0}: Tags changed to {1}'.format(snapshot['SnapshotId'], new_tags))
ec2.create_tags(Resources=[snapshot['SnapshotId']], Tags=ansible_dict_to_boto3_tag_list(new_tags))
for snapshot in snapshots.values():
if 'Used' not in snapshot:
cur_tags = boto3_tag_list_to_ansible_dict(snapshot.get('Tags', []))
name = cur_tags.get('Name', snapshot['SnapshotId'])
if not name.startswith('UNUSED'):
logger.warning('{0} Unused!'.format(snapshot['SnapshotId']))
cur_tags['Name'] = 'UNUSED ' + name
ec2.create_tags(Resources=[snapshot['SnapshotId']], Tags=ansible_dict_to_boto3_tag_list(cur_tags))
def tag_volumes():
volumes = {}
for response in ec2.get_paginator('describe_volumes').paginate():
volumes.update([(volume['VolumeId'], volume) for volume in response['Volumes']])
for response in ec2.get_paginator('describe_instances').paginate():
for reservation in response['Reservations']:
for instance in reservation['Instances']:
tags = boto3_tag_list_to_ansible_dict(instance.get('Tags', []))
for device in instance['BlockDeviceMappings']:
volume = volumes[device['Ebs']['VolumeId']]
volume['Used'] = True
cur_tags = boto3_tag_list_to_ansible_dict(volume.get('Tags', []))
new_tags = copy.deepcopy(cur_tags)
new_tags.update(tags)
new_tags['Name'] += ' ' + device['DeviceName']
if new_tags != cur_tags:
logger.info('{0} Tags changed to {1}'.format(volume['VolumeId'], new_tags))
ec2.create_tags(Resources=[volume['VolumeId']], Tags=ansible_dict_to_boto3_tag_list(new_tags))
for volume in volumes.values():
if 'Used' not in volume:
cur_tags = boto3_tag_list_to_ansible_dict(volume.get('Tags', []))
name = cur_tags.get('Name', volume['VolumeId'])
if not name.startswith('UNUSED'):
logger.warning('{0} Unused!'.format(volume['VolumeId']))
cur_tags['Name'] = 'UNUSED ' + name
ec2.create_tags(Resources=[volume['VolumeId']], Tags=ansible_dict_to_boto3_tag_list(cur_tags))
def tag_everything():
tag_snapshots()
tag_volumes()
def boto3_tag_list_to_ansible_dict(tags_list):
tags_dict = {}
for tag in tags_list:
if 'key' in tag and not tag['key'].startswith('aws:'):
tags_dict[tag['key']] = tag['value']
elif 'Key' in tag and not tag['Key'].startswith('aws:'):
tags_dict[tag['Key']] = tag['Value']
return tags_dict
def ansible_dict_to_boto3_tag_list(tags_dict):
tags_list = []
for k, v in tags_dict.items():
tags_list.append({'Key': k, 'Value': v})
return tags_list
def handler(event, context):
tag_everything()
if __name__ == '__main__':
tag_everything()
@vasanthlebara
Copy link

Is there a way to modify this so that it tags Snaps created manually or from the Lifecycle Manager, so not just AMIs? As currently any others just get tagged with "UNUSED snap-xxxxxxxx"

@migdotcom
Copy link

In risk of outing myself as a beginner. How is this code ran in AWS? I am not sure how to implement it.

@brandond
Copy link
Author

@migdotcom you run it on a host that has AWS credentials available. Could be an EC2 instance with an instance role, could be a dev/ops workstation with a CLI profile holding credentials.

@migdotcom
Copy link

@migdotcom you run it on a host that has AWS credentials available. Could be an EC2 instance with an instance role, could be a dev/ops workstation with a CLI profile holding credentials.

Thanks for answering that as I didn't know that I didn't know that. I was going to use it as a lambda function. I meant more in the side of inputs and env I have a batch of ec2 snapshots. I see boto3_tag_list_to_ansible_dict(tags_list) how do I define this list.

@brandond
Copy link
Author

You definitely could use it in a lambda function if you wanted to. This just runs against all the snapshots in an account. You don't have to feed it any tags, it looks them up from hosts and volumes.

@harishsd1998
Copy link

Hello Brandon, is there a way to display ec2 tags in two separate columns with key in one column and value in another column. Could you please help me with that?

@brandond
Copy link
Author

@harishsd1998 yes but that's a pretty basic task that you could do with awscli or boto3, unrelated to the functionality here?

@harishsd1998
Copy link

harishsd1998 commented May 20, 2021

Actually I used boto3 and printed Ec2 instances and the tags in excel format but tags column contain both key and value in a list format. I want to print that key and value in separate columns? Can you please help me with that?

@ro6it
Copy link

ro6it commented Aug 6, 2021

Hi Brandon- There are a lot of snapshots and volumes that I need to tag, I tried your script using lambda but it didn't work.

Function Logs
START RequestId: 37b948f9-d89d-4782-bb3b-5a29b4197967 Version: $LATEST
[ERROR] Runtime.HandlerNotFound: Handler 'lambda_handler' missing on module 'lambda_function'
Traceback (most recent call last):END RequestId: 37b948f9-d89d-4782-bb3b-5a29b4197967
REPORT RequestId: 37b948f9-d89d-4782-bb3b-5a29b4197967 Duration: 1.24 ms Billed Duration: 2 ms Memory Size: 300 MB Max Memory Used: 85 MB

@brandond
Copy link
Author

brandond commented Aug 6, 2021

This is like 4 years old and uses python 2.7; I'm pretty sure it needs updating but I don't do AWS infra any more (I work for SUSE Rancher now) so I'm unlikely to fix it myself. Feel free to fork and modify!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment