Skip to content

Instantly share code, notes, and snippets.

@bkozora
Last active October 22, 2023 20:49
  • Star 46 You must be signed in to star a gist
  • Fork 64 You must be signed in to fork a gist
Star You must be signed in to star a gist
Save bkozora/724e01903a9ad481d21e to your computer and use it in GitHub Desktop.
AWS Lambda AMI Backups
# Automated AMI Backups
#
# @author Bobby Kozora
#
# This script will search for all instances having a tag with the name "backup"
# and value "Backup" on it. As soon as we have the instances list, we loop
# through each instance
# and create an AMI of it. Also, it will look for a "Retention" tag key which
# will be used as a retention policy number in days. If there is no tag with
# that name, it will use a 7 days default value for each AMI.
#
# After creating the AMI it creates a "DeleteOn" tag on the AMI indicating when
# it will be deleted using the Retention value and another Lambda function
import boto3
import collections
import datetime
import sys
import pprint
ec = boto3.client('ec2')
#image = ec.Image('id')
def lambda_handler(event, context):
reservations = ec.describe_instances(Filters=[
{
'Name': 'tag-key',
'Values': ['backup', 'Backup']
},
]).get('Reservations', [])
instances = sum([[i for i in r['Instances']] for r in reservations], [])
print("Found %d instances that need backing up" % len(instances))
to_tag = collections.defaultdict(list)
for instance in instances:
try:
retention_days = [
int(t.get('Value')) for t in instance['Tags']
if t['Key'] == 'Retention'
][0]
except IndexError:
retention_days = 7
#for dev in instance['BlockDeviceMappings']:
# if dev.get('Ebs', None) is None:
# continue
# vol_id = dev['Ebs']['VolumeId']
# print "Found EBS volume %s on instance %s" % (
# vol_id, instance['InstanceId'])
#snap = ec.create_snapshot(
# VolumeId=vol_id,
#)
#create_image(instance_id, name, description=None, no_reboot=False, block_device_mapping=None, dry_run=False)
# DryRun, InstanceId, Name, Description, NoReboot, BlockDeviceMappings
create_time = datetime.datetime.now()
create_fmt = create_time.strftime('%Y-%m-%d')
AMIid = ec.create_image(
InstanceId=instance['InstanceId'],
Name="Lambda - " + instance['InstanceId'] + " from " +
create_fmt,
Description="Lambda created AMI of instance " +
instance['InstanceId'] + " from " + create_fmt,
NoReboot=True,
DryRun=False)
pprint.pprint(instance)
#sys.exit()
#break
#to_tag[retention_days].append(AMIid)
to_tag[retention_days].append(AMIid['ImageId'])
print("Retaining AMI %s of instance %s for %d days" % (
AMIid['ImageId'],
instance['InstanceId'],
retention_days,
))
print(to_tag.keys())
for retention_days in to_tag.keys():
delete_date = datetime.date.today() + datetime.timedelta(
days=retention_days)
delete_fmt = delete_date.strftime('%m-%d-%Y')
print("Will delete %d AMIs on %s" %
(len(to_tag[retention_days]), delete_fmt))
#break
ec.create_tags(Resources=to_tag[retention_days],
Tags=[
{
'Key': 'DeleteOn',
'Value': delete_fmt
},
])
@mcalr3
Copy link

mcalr3 commented Apr 9, 2018

Thanks to @bkozora for creating these scripts. They are both performing the desired function.

Other than outputting the result of each Lambda execution to Cloudwatch logs, is there any way to output specific lines in the log to an SNS topic or send by email using SES?

I am looking for a daily email with a summary of AMIs created, along with any errors in the log.
I have created a Cloudwatch alarm which sends an email if any error in running the lambda function but this only sends if the function itself fails, not if any resulting actions performed such as snapshots fail.

@sstalon
Copy link

sstalon commented Apr 10, 2018

For those who want to add the Instance Name into the AMI this is the code I used.
snapshot_name = 'N/A'
if 'Tags' in instance:
for tags in instance['Tags']:
if tags["Key"] == 'Name':
snapshot_name = tags["Value"]
print "Tagging snapshot with Name: %s" % (snapshot_name)

Than

I moddifed:
AMIid = ec.create_image(InstanceId=instance['InstanceId'], Name="Lambda - " + instance['InstanceId'] + " from " + create_fmt, Description="Lambda created AMI of instance " + instance['InstanceId'] + " from " + create_fmt, NoReboot=True, DryRun=False)

and added snapshot_name into that. Depending where you want to added it example

"Lambda - " + instance['InstanceId'] + " " + snapshot_name + " from " + create_fmt etc etc...

@sstalon
Copy link

sstalon commented Apr 10, 2018

Does anyone know how would would tag the individual snapshots that are attached to the AMI with names or descriptions?

@ramesher
Copy link

I want to take all ec2 intance tags and add into AMI when it takes. Is it possible?

@sstalon
Copy link

sstalon commented Apr 11, 2018

@ramesher
Yes, look at my post above. Use that as example to create the tags you want,
and than at the end you need to modify this line to add your additional tags accordingly

ec.create_tags(
Resources=to_tag[retention_days],
Tags=[
{'Key': 'DeleteOn', 'Value': delete_fmt},
]
)

@sstalon
Copy link

sstalon commented Apr 11, 2018

@ramesher

Or to make it simple this is a example

ec.create_tags(
Resources=to_tag[retention_days],
Tags=[
{'Key': 'DeleteOn', 'Value': delete_fmt},
{'Key': 'Account', 'Value': 'Gov'},
{'Key': 'Environment', 'Value': 'Test'}
]
)

@ramesher
Copy link

Thank you. Let me check and update you. Is it possible to take instance type and SG also?

@sstalon
Copy link

sstalon commented Apr 12, 2018

@ramesher
of course, you just have to figure out how to query that into a variable.
like example for me i want a name to my AMI.
so so i just use my code of

instance_name = 'N/A'
if 'Tags' in instance:
for tags in instance['Tags']:
if tags["Key"] == 'Name':
instance_name = tags["Value"]
print "Tagging snapshot with Name: %s" % (instance_name)

and than modify my tag code to have this

ec.create_tags(
Resources=to_tag[retention_days],
Tags=[
{'Key': 'DeleteOn', 'Value': delete_fmt},
{'Key': 'Account', 'Value': 'Gov'},
{'Key': 'Environment', 'Value': 'Test'},
{'Key': 'Name', 'Value': instance_name}
]
)

@mcalr3
Copy link

mcalr3 commented Apr 17, 2018

Could someone please tell me how I send an email notification with the exection result of this function, or just some sort of output to show the backup went well?

EDIT: I figred out how to do this using Cloudwatch Logs and creating a Metric Filter for the logs looking for specific keywords in the script that I was interested in Alerting on. For example, "error". Now I don't care if the whole log gets sent by email or not because I can log in and check the logs right from cloudwatch.

@mcalr3
Copy link

mcalr3 commented Jun 4, 2018

In relation to my comment on the Cleanup script where it was timing out if you were looping through too many instances, here is the code I inserted into this Backup script in order to add another Tag with InstanceID to the AMI, which allows us to filter on this value in the cleanup script to allow much better performance.

Replace lines 78 - 82 with the following:

            print "Creating AMI %s of instance %s and retaining for %d days" % (
                AMIid['ImageId'],
                instance['InstanceId'],
                retention_days,
            )
            print "--------"
            amiid = AMIid['ImageId']
            instanceid = instance['InstanceId']
            tag_image = ec2.Image(amiid)
            tag_image.create_tags(
                Tags=[
                    {'Key': 'InstanceID', 'Value': instanceid},
                ]
            )

@AndrewFarley
Copy link

AndrewFarley commented Jun 5, 2018

Hey guys... so... I saw this and all your struggling and it kinda made me weep for the internet as a whole. So, I've been needing this a while... I finally got around to it and made it into a micro example/open source serverless stack. It's here on my github, called AWS Automated Daily Snapshots. It will...

  • Automatically scan through EVERY AWS region (by default, you can also narrow the list to speed it up)
  • Will automatically deploy to your AWS account with restrictive permissions and set a CloudWatch event to run daily
  • Will automatically backup instances to AMIs and then delete those AMIs as they become stale
  • Will preserve ALL tags that the instance had, on the AMI (and adding a few for our purposes, to rotate)
  • Has a bit of error checking... but it could have more. :P Haven't hit any errors yet that I'm not catching but if you guys find any, let me know I'll gladly fix it.
  • And is totally open source and simple as heck, so feel free to hack on it yourselves. Forks/MRs welcome.
  • Is very loosely based on the logic you guys had here above, but really rewritten from the ground up, as you can tell. Some of the flaws in the linked delete script everyone had is fixed in my project.

Enjoy y'all! :)

@Benom-10
Copy link

Benom-10 commented Jun 6, 2018

I have a windows instance,
how do I have it stop the instance before it creates the AMI and start afterward?
Also does it need any waits?

@charterresources
Copy link

@AndrewFarley
Thank you for your contribution. It worked like a charm.
two questions:

  • How can we make sure the AMI-Name timestamps are based on our current timezone?
  • How can we tag the snapshots using original volume tags

@mcalr3
Copy link

mcalr3 commented Jun 26, 2018

@AndrewFarley
Thanks for your project. Have you tried it in an account with 80+ instances? I am using @bkozora 's code which I tweaked and find that each function (creation then cleanup) runs for at least a minute each. Sometimes it fails due to boto3 errors (max retries reached) but Lambda re-runs it automatically.

Just want to know how scalable your serverless app is.

@cmcconnell1
Copy link

Thanks @AndrewFarley (and @bkozora for this gist). I just implemented the https://github.com/AndrewFarley/AWSAutomatedDailyInstanceAMISnapshots project to replace a previous CM-based solution we had, and it works great! I just submitted a P/R for adding to the README with examples of scheduling the daily backups via cron, and aws cli query filtering on tags, to help folks visualize and manage AMI's, etc.
This project is a gem and saved me a lot of time reinventing another wheel. Thanks again!

@tmblue
Copy link

tmblue commented Aug 1, 2018

Anyone tweak this or the addition that @cmcconnell1 made to do snapshots. Not AMI's but of volumes, to replace the sensitive and kind of clunky AWS CLI ec2-automate-backup ?

@DJ-Laker
Copy link

I've just tried implementing this script for the first time and am getting the following error - can anyone provide some guidance please?


09:34:25
START RequestId: 568053dd-cf94-11e8-8b3a-a7435442769d Version: $LATEST

09:34:25
module initialization error: name 'instances' is not defined

09:34:25
END RequestId: 568053dd-cf94-11e8-8b3a-a7435442769d

09:34:25
REPORT RequestId: 568053dd-cf94-11e8-8b3a-a7435442769d Duration: 557.09 ms Billed Duration: 600 ms Memory Size: 128 MB Max Memory Used: 56 MB

09:34:25
module initialization error name 'instances' is not defined

@psipher
Copy link

psipher commented Oct 31, 2018

@nehas19
Copy link

nehas19 commented Feb 21, 2019

some created images are not getting tags,
so , they are not deleting using delete function.

@Phanindrapavan
Copy link

Hi i need instance names tagging with AMI as following

AMIid = ec.create_image(InstanceId=instance['InstanceId'], Name="Lambda - " + instance_Name + " from " + create_fmt, Description="Lambda created AMI of instance " + instance_Name + " from " + create_fmt, NoReboot=True, DryRun=False)

Can any one help me?
@sstalon your modification mentioned @ramesher is not working

@prasantaku
Copy link

prasantaku commented Aug 13, 2019

My execution failed. Can u help me
Response:
{
"errorMessage": "Syntax error in module 'lambda_function': invalid syntax (lambda_function.py, line 39)",
"errorType": "Runtime.UserCodeSyntaxError",
"stackTrace": [
" File "/var/task/lambda_function.py" Line 39\n print "Found %d instances that need backing up" % len(instances)\n"
]
}

Request ID:
"f95badef-b2be-4716-8f57-ac650e051c03"

Function Logs:
START RequestId: f95badef-b2be-4716-8f57-ac650e051c03 Version: $LATEST
[ERROR] Runtime.UserCodeSyntaxError: Syntax error in module 'lambda_function': invalid syntax (lambda_function.py, line 39)
Traceback (most recent call last):
  File "/var/task/lambda_function.py" Line 39
        print "Found %d instances that need backing up" % len(instances)
END RequestId: f95badef-b2be-4716-8f57-ac650e051c03
REPORT RequestId: f95badef-b2be-4716-8f57-ac650e051c03 Duration: 36.79 ms Billed Duration: 100 ms Memory Size: 128 MB Max Memory Used: 55 MB

@prasantaku
Copy link

execution failed plz help me

Response:
{
"errorMessage": "Syntax error in module 'lambda_function': invalid syntax (lambda_function.py, line 39)",
"errorType": "Runtime.UserCodeSyntaxError",
"stackTrace": [
" File "/var/task/lambda_function.py" Line 39\n print "Found %d instances that need backing up" % len(instances)\n"
]
}

Request ID:
"f95badef-b2be-4716-8f57-ac650e051c03"

Function Logs:
START RequestId: f95badef-b2be-4716-8f57-ac650e051c03 Version: $LATEST
[ERROR] Runtime.UserCodeSyntaxError: Syntax error in module 'lambda_function': invalid syntax (lambda_function.py, line 39)
Traceback (most recent call last):
  File "/var/task/lambda_function.py" Line 39
        print "Found %d instances that need backing up" % len(instances)
END RequestId: f95badef-b2be-4716-8f57-ac650e051c03
REPORT RequestId: f95badef-b2be-4716-8f57-ac650e051c03 Duration: 36.79 ms Billed Duration: 100 ms Memory Size: 128 MB Max Memory Used: 55 MB

@sstalon
Copy link

sstalon commented Sep 16, 2019

execution failed plz help me

Response:
{
"errorMessage": "Syntax error in module 'lambda_function': invalid syntax (lambda_function.py, line 39)",
"errorType": "Runtime.UserCodeSyntaxError",
"stackTrace": [
" File "/var/task/lambda_function.py" Line 39\n print "Found %d instances that need backing up" % len(instances)\n"
]
}

Request ID:
"f95badef-b2be-4716-8f57-ac650e051c03"

Function Logs:
START RequestId: f95badef-b2be-4716-8f57-ac650e051c03 Version: $LATEST
[ERROR] Runtime.UserCodeSyntaxError: Syntax error in module 'lambda_function': invalid syntax (lambda_function.py, line 39)
Traceback (most recent call last):
  File "/var/task/lambda_function.py" Line 39
        print "Found %d instances that need backing up" % len(instances)
END RequestId: f95badef-b2be-4716-8f57-ac650e051c03
REPORT RequestId: f95badef-b2be-4716-8f57-ac650e051c03 Duration: 36.79 ms Billed Duration: 100 ms Memory Size: 128 MB Max Memory Used: 55 MB

Your probably using higher than python 2.7. Looks like his code only works on Python 2.7.

@bkozora
Copy link
Author

bkozora commented Sep 18, 2019

I've updated the code to be syntactically compliant with Python 3, only having to wrap print in parenthesis. This will prob be the last update I make, however. AWS has evolved a great deal and now offers automated backups using Systems Manager.

https://docs.aws.amazon.com/en_pv/systems-manager/latest/userguide/automation-aws-createimage.html

https://console.aws.amazon.com/systems-manager/documents/AWS-CreateImage/content?region=us-east-1

@raparty
Copy link

raparty commented Dec 31, 2019

The script works great. I want to send an email of the list of instances for which the AMI's are created. How can I get that.

@kenlee-memberson
Copy link

hi bkozora,
Firstly let me thank you for your contribution. I have been using this script for auto AMI backup since a year ago and it works great. However since python 2 is ending support, I have to change to use the latest updated script (python 3.8). The Lambda runs and there is no specific error, I'm only getting the below message from cloudwatch but the AMI backups will not appear. Can you please help?


06:45:10
START RequestId: e87b6746-8746-4bac-ab31-6afda9ee6515 Version: $LATEST

06:45:10
Found 14 instances that need backing up

06:45:10
dict_keys([])

@sivahome
Copy link

Hi Guys,
I took reference of this script and did ami deletion based on the tag .
But the requirement is Based on tag as well , that Tagged instance backup should be retained 30 days if more than 30 days it should delete.

Could you please help me here with logic

Code below
++++++++++
import os
import re
from datetime import datetime, timedelta
import boto3
import collections
import datetime
import logging
import time
import sys

ec = boto3.client('ec2',)
ec2 = boto3.resource('ec2',)
images = ec2.images.filter(Owners=["self"])

def lambda_handler(event, context):

to_tag = collections.defaultdict(list)
date = datetime.datetime.now()
date_fmt = date.strftime('%Y-%m-%d')

imagesList = []

for image in images:

       try:
            if image.tags is not None:
                deletion_date = [
                    t.get('Value') for t in image.tags
                    if t['Key'] == 'backup'
                ][0]
                imagesList.append(image.id)
                
                
       except IndexError:
           deletion_date = False
           delete_date = False

       today_time = datetime.datetime.now().strftime('%m-%d-%Y')
        # today_fmt = today_time.strftime('%m-%d-%Y')
       today_date = time.strptime(today_time, '%m-%d-%Y')

                     
       
print ("=============")

print ("About to process the following AMIs:")
print ('imagesList')

myAccount = boto3.client('sts').get_caller_identity()['Account']

snapshots = ec.describe_snapshots(MaxResults=1000, OwnerIds=[myAccount])['Snapshots']

# loop through list of image IDs
for image in imagesList:
    try:
        print ("deregistering image %s" % image)
        amiResponse = ec.deregister_image(
        DryRun=False,
        ImageId=image,
       )
    except ClientError:
        print ("ClientError, likely AMI not found")
        continue

    for snapshot in snapshots:
        if snapshot['Description'].find(image) > 0:
            snap = ec.delete_snapshot(SnapshotId=snapshot['SnapshotId'])
            print ("Deleting snapshot " + snapshot['SnapshotId'])

print ("-------------")
+++++++++++++++

@ssreddygadiko
Copy link

How to tag snapshot by using ami id created by this script

@shivangitripathi
Copy link

Thank you for the nice script. :)

@phani44
Copy link

phani44 commented Jun 8, 2021

Hi thanks for this nice script. I have made a small enhancement on the script, so it also supports instances with multiple EBS volumes attached.

On line 42 i have added a new empty list, that will hold all instances that where already done :
inDone = []

And the line 67 should be surrounded by a check. Just replace the line 67 with:

            if str(instance['InstanceId']) not in inDone:

              AMIid = ec.create_image(InstanceId=instance['InstanceId'], Name="Lambda - " + instance['InstanceId'] + " from " + create_fmt, Description="Lambda created AMI of instance " + instance['InstanceId'] + " from " + create_fmt, NoReboot=True, DryRun=False)

              inDone.insert(0,str(instance['InstanceId']))

              print "Created AMI %s of instance %s " % (AMIid['ImageId'], instance['InstanceId'])

            else:

              print "We already got an AMI of instance %s " % (instance['InstanceId'])

Thanks again!
Regards, Frank

Hello Frank,

Can you paste me the full script along with modifications you made

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment