AWS Lambda AMI Backups

lambdaAMIBackups.py

# Automated AMI Backups
#
# @author Bobby Kozora
#
# This script will search for all instances having a tag with the name "backup"
# and value "Backup" on it. As soon as we have the instances list, we loop
# through each instance
# and create an AMI of it. Also, it will look for a "Retention" tag key which
# will be used as a retention policy number in days. If there is no tag with
# that name, it will use a 7 days default value for each AMI.
#
# After creating the AMI it creates a "DeleteOn" tag on the AMI indicating when
# it will be deleted using the Retention value and another Lambda function

import boto3
import collections
import datetime
import sys
import pprint

ec = boto3.client('ec2')
#image = ec.Image('id')


def lambda_handler(event, context):

    reservations = ec.describe_instances(Filters=[
        {
            'Name': 'tag-key',
            'Values': ['backup', 'Backup']
        },
    ]).get('Reservations', [])

    instances = sum([[i for i in r['Instances']] for r in reservations], [])

    print("Found %d instances that need backing up" % len(instances))

    to_tag = collections.defaultdict(list)

    for instance in instances:
        try:
            retention_days = [
                int(t.get('Value')) for t in instance['Tags']
                if t['Key'] == 'Retention'
            ][0]
        except IndexError:
            retention_days = 7

            #for dev in instance['BlockDeviceMappings']:
            #    if dev.get('Ebs', None) is None:
            #        continue
            #    vol_id = dev['Ebs']['VolumeId']
            #    print "Found EBS volume %s on instance %s" % (
            #        vol_id, instance['InstanceId'])

            #snap = ec.create_snapshot(
            #    VolumeId=vol_id,
            #)

            #create_image(instance_id, name, description=None, no_reboot=False, block_device_mapping=None, dry_run=False)
            # DryRun, InstanceId, Name, Description, NoReboot, BlockDeviceMappings
            create_time = datetime.datetime.now()
            create_fmt = create_time.strftime('%Y-%m-%d')

            AMIid = ec.create_image(
                InstanceId=instance['InstanceId'],
                Name="Lambda - " + instance['InstanceId'] + " from " +
                create_fmt,
                Description="Lambda created AMI of instance " +
                instance['InstanceId'] + " from " + create_fmt,
                NoReboot=True,
                DryRun=False)

            pprint.pprint(instance)
            #sys.exit()
            #break

            #to_tag[retention_days].append(AMIid)

            to_tag[retention_days].append(AMIid['ImageId'])

            print("Retaining AMI %s of instance %s for %d days" % (
                AMIid['ImageId'],
                instance['InstanceId'],
                retention_days,
            ))

    print(to_tag.keys())

    for retention_days in to_tag.keys():
        delete_date = datetime.date.today() + datetime.timedelta(
            days=retention_days)
        delete_fmt = delete_date.strftime('%m-%d-%Y')
        print("Will delete %d AMIs on %s" %
              (len(to_tag[retention_days]), delete_fmt))

        #break

        ec.create_tags(Resources=to_tag[retention_days],
                       Tags=[
                           {
                               'Key': 'DeleteOn',
                               'Value': delete_fmt
                           },
                       ])

Thanks to @bkozora for creating these scripts. They are both performing the desired function.

Other than outputting the result of each Lambda execution to Cloudwatch logs, is there any way to output specific lines in the log to an SNS topic or send by email using SES?

I am looking for a daily email with a summary of AMIs created, along with any errors in the log.
I have created a Cloudwatch alarm which sends an email if any error in running the lambda function but this only sends if the function itself fails, not if any resulting actions performed such as snapshots fail.

For those who want to add the Instance Name into the AMI this is the code I used.
snapshot_name = 'N/A'
if 'Tags' in instance:
for tags in instance['Tags']:
if tags["Key"] == 'Name':
snapshot_name = tags["Value"]
print "Tagging snapshot with Name: %s" % (snapshot_name)

Than

I moddifed:
AMIid = ec.create_image(InstanceId=instance['InstanceId'], Name="Lambda - " + instance['InstanceId'] + " from " + create_fmt, Description="Lambda created AMI of instance " + instance['InstanceId'] + " from " + create_fmt, NoReboot=True, DryRun=False)

and added snapshot_name into that. Depending where you want to added it example

"Lambda - " + instance['InstanceId'] + " " + snapshot_name + " from " + create_fmt etc etc...

Does anyone know how would would tag the individual snapshots that are attached to the AMI with names or descriptions?

I want to take all ec2 intance tags and add into AMI when it takes. Is it possible?

@ramesher
Yes, look at my post above. Use that as example to create the tags you want,
and than at the end you need to modify this line to add your additional tags accordingly

ec.create_tags(
Resources=to_tag[retention_days],
Tags=[
{'Key': 'DeleteOn', 'Value': delete_fmt},
]
)

@ramesher

Or to make it simple this is a example

ec.create_tags(
Resources=to_tag[retention_days],
Tags=[
{'Key': 'DeleteOn', 'Value': delete_fmt},
{'Key': 'Account', 'Value': 'Gov'},
{'Key': 'Environment', 'Value': 'Test'}
]
)

Thank you. Let me check and update you. Is it possible to take instance type and SG also?

@ramesher
of course, you just have to figure out how to query that into a variable.
like example for me i want a name to my AMI.
so so i just use my code of

instance_name = 'N/A'
if 'Tags' in instance:
for tags in instance['Tags']:
if tags["Key"] == 'Name':
instance_name = tags["Value"]
print "Tagging snapshot with Name: %s" % (instance_name)

and than modify my tag code to have this

ec.create_tags(
Resources=to_tag[retention_days],
Tags=[
{'Key': 'DeleteOn', 'Value': delete_fmt},
{'Key': 'Account', 'Value': 'Gov'},
{'Key': 'Environment', 'Value': 'Test'},
{'Key': 'Name', 'Value': instance_name}
]
)

Could someone please tell me how I send an email notification with the exection result of this function, or just some sort of output to show the backup went well?

EDIT: I figred out how to do this using Cloudwatch Logs and creating a Metric Filter for the logs looking for specific keywords in the script that I was interested in Alerting on. For example, "error". Now I don't care if the whole log gets sent by email or not because I can log in and check the logs right from cloudwatch.

In relation to my comment on the Cleanup script where it was timing out if you were looping through too many instances, here is the code I inserted into this Backup script in order to add another Tag with InstanceID to the AMI, which allows us to filter on this value in the cleanup script to allow much better performance.

Replace lines 78 - 82 with the following:

            print "Creating AMI %s of instance %s and retaining for %d days" % (
                AMIid['ImageId'],
                instance['InstanceId'],
                retention_days,
            )
            print "--------"
            amiid = AMIid['ImageId']
            instanceid = instance['InstanceId']
            tag_image = ec2.Image(amiid)
            tag_image.create_tags(
                Tags=[
                    {'Key': 'InstanceID', 'Value': instanceid},
                ]
            )

Hey guys... so... I saw this and all your struggling and it kinda made me weep for the internet as a whole. So, I've been needing this a while... I finally got around to it and made it into a micro example/open source serverless stack. It's here on my github, called AWS Automated Daily Snapshots. It will...

• Automatically scan through EVERY AWS region (by default, you can also narrow the list to speed it up)
• Will automatically deploy to your AWS account with restrictive permissions and set a CloudWatch event to run daily
• Will automatically backup instances to AMIs and then delete those AMIs as they become stale
• Will preserve ALL tags that the instance had, on the AMI (and adding a few for our purposes, to rotate)
• Has a bit of error checking... but it could have more. :P Haven't hit any errors yet that I'm not catching but if you guys find any, let me know I'll gladly fix it.
• And is totally open source and simple as heck, so feel free to hack on it yourselves. Forks/MRs welcome.
• Is very loosely based on the logic you guys had here above, but really rewritten from the ground up, as you can tell. Some of the flaws in the linked delete script everyone had is fixed in my project.

Enjoy y'all! :)

I have a windows instance,
how do I have it stop the instance before it creates the AMI and start afterward?
Also does it need any waits?

@AndrewFarley
Thank you for your contribution. It worked like a charm.
two questions:

• How can we make sure the AMI-Name timestamps are based on our current timezone?
• How can we tag the snapshots using original volume tags

@AndrewFarley
Thanks for your project. Have you tried it in an account with 80+ instances? I am using @bkozora 's code which I tweaked and find that each function (creation then cleanup) runs for at least a minute each. Sometimes it fails due to boto3 errors (max retries reached) but Lambda re-runs it automatically.

Just want to know how scalable your serverless app is.

Thanks @AndrewFarley (and @bkozora for this gist). I just implemented the https://github.com/AndrewFarley/AWSAutomatedDailyInstanceAMISnapshots project to replace a previous CM-based solution we had, and it works great! I just submitted a P/R for adding to the README with examples of scheduling the daily backups via cron, and aws cli query filtering on tags, to help folks visualize and manage AMI's, etc.
This project is a gem and saved me a lot of time reinventing another wheel. Thanks again!

Anyone tweak this or the addition that @cmcconnell1 made to do snapshots. Not AMI's but of volumes, to replace the sensitive and kind of clunky AWS CLI ec2-automate-backup ?

I've just tried implementing this script for the first time and am getting the following error - can anyone provide some guidance please?


09:34:25
START RequestId: 568053dd-cf94-11e8-8b3a-a7435442769d Version: $LATEST

09:34:25
module initialization error: name 'instances' is not defined

09:34:25
END RequestId: 568053dd-cf94-11e8-8b3a-a7435442769d

09:34:25
REPORT RequestId: 568053dd-cf94-11e8-8b3a-a7435442769d Duration: 557.09 ms Billed Duration: 600 ms Memory Size: 128 MB Max Memory Used: 56 MB

09:34:25
module initialization error name 'instances' is not defined

@DJ-Laker
Copy the raw text to preserve the indentation.
https://stackoverflow.com/questions/49618760/aws-lambda-error-module-initialization-error-name-dynamodb-is-not-defined-w

some created images are not getting tags,
so , they are not deleting using delete function.

Hi i need instance names tagging with AMI as following

AMIid = ec.create_image(InstanceId=instance['InstanceId'], Name="Lambda - " + instance_Name + " from " + create_fmt, Description="Lambda created AMI of instance " + instance_Name + " from " + create_fmt, NoReboot=True, DryRun=False)

Can any one help me?
@sstalon your modification mentioned @ramesher is not working

My execution failed. Can u help me
Response:
{
"errorMessage": "Syntax error in module 'lambda_function': invalid syntax (lambda_function.py, line 39)",
"errorType": "Runtime.UserCodeSyntaxError",
"stackTrace": [
" File "/var/task/lambda_function.py" Line 39\n print "Found %d instances that need backing up" % len(instances)\n"
]
}

Request ID:
"f95badef-b2be-4716-8f57-ac650e051c03"

Function Logs:
START RequestId: f95badef-b2be-4716-8f57-ac650e051c03 Version: $LATEST
[ERROR] Runtime.UserCodeSyntaxError: Syntax error in module 'lambda_function': invalid syntax (lambda_function.py, line 39)
Traceback (most recent call last):
  File "/var/task/lambda_function.py" Line 39
        print "Found %d instances that need backing up" % len(instances)
END RequestId: f95badef-b2be-4716-8f57-ac650e051c03
REPORT RequestId: f95badef-b2be-4716-8f57-ac650e051c03 Duration: 36.79 ms Billed Duration: 100 ms Memory Size: 128 MB Max Memory Used: 55 MB

execution failed plz help me

Response:
{
"errorMessage": "Syntax error in module 'lambda_function': invalid syntax (lambda_function.py, line 39)",
"errorType": "Runtime.UserCodeSyntaxError",
"stackTrace": [
" File "/var/task/lambda_function.py" Line 39\n print "Found %d instances that need backing up" % len(instances)\n"
]
}

Request ID:
"f95badef-b2be-4716-8f57-ac650e051c03"

Function Logs:
START RequestId: f95badef-b2be-4716-8f57-ac650e051c03 Version: $LATEST
[ERROR] Runtime.UserCodeSyntaxError: Syntax error in module 'lambda_function': invalid syntax (lambda_function.py, line 39)
Traceback (most recent call last):
  File "/var/task/lambda_function.py" Line 39
        print "Found %d instances that need backing up" % len(instances)
END RequestId: f95badef-b2be-4716-8f57-ac650e051c03
REPORT RequestId: f95badef-b2be-4716-8f57-ac650e051c03 Duration: 36.79 ms Billed Duration: 100 ms Memory Size: 128 MB Max Memory Used: 55 MB

execution failed plz help me

Response:
{
"errorMessage": "Syntax error in module 'lambda_function': invalid syntax (lambda_function.py, line 39)",
"errorType": "Runtime.UserCodeSyntaxError",
"stackTrace": [
" File "/var/task/lambda_function.py" Line 39\n print "Found %d instances that need backing up" % len(instances)\n"
]
}

Request ID:
"f95badef-b2be-4716-8f57-ac650e051c03"

Function Logs:
START RequestId: f95badef-b2be-4716-8f57-ac650e051c03 Version: $LATEST
[ERROR] Runtime.UserCodeSyntaxError: Syntax error in module 'lambda_function': invalid syntax (lambda_function.py, line 39)
Traceback (most recent call last):
  File "/var/task/lambda_function.py" Line 39
        print "Found %d instances that need backing up" % len(instances)
END RequestId: f95badef-b2be-4716-8f57-ac650e051c03
REPORT RequestId: f95badef-b2be-4716-8f57-ac650e051c03 Duration: 36.79 ms Billed Duration: 100 ms Memory Size: 128 MB Max Memory Used: 55 MB

Your probably using higher than python 2.7. Looks like his code only works on Python 2.7.

I've updated the code to be syntactically compliant with Python 3, only having to wrap print in parenthesis. This will prob be the last update I make, however. AWS has evolved a great deal and now offers automated backups using Systems Manager.

https://docs.aws.amazon.com/en_pv/systems-manager/latest/userguide/automation-aws-createimage.html

https://console.aws.amazon.com/systems-manager/documents/AWS-CreateImage/content?region=us-east-1

The script works great. I want to send an email of the list of instances for which the AMI's are created. How can I get that.

hi bkozora,
Firstly let me thank you for your contribution. I have been using this script for auto AMI backup since a year ago and it works great. However since python 2 is ending support, I have to change to use the latest updated script (python 3.8). The Lambda runs and there is no specific error, I'm only getting the below message from cloudwatch but the AMI backups will not appear. Can you please help?


06:45:10
START RequestId: e87b6746-8746-4bac-ab31-6afda9ee6515 Version: $LATEST

06:45:10
Found 14 instances that need backing up

06:45:10
dict_keys([])

Hi Guys,
I took reference of this script and did ami deletion based on the tag .
But the requirement is Based on tag as well , that Tagged instance backup should be retained 30 days if more than 30 days it should delete.

Could you please help me here with logic

Code below
++++++++++
import os
import re
from datetime import datetime, timedelta
import boto3
import collections
import datetime
import logging
import time
import sys

ec = boto3.client('ec2',)
ec2 = boto3.resource('ec2',)
images = ec2.images.filter(Owners=["self"])

def lambda_handler(event, context):

to_tag = collections.defaultdict(list)
date = datetime.datetime.now()
date_fmt = date.strftime('%Y-%m-%d')

imagesList = []

for image in images:

       try:
            if image.tags is not None:
                deletion_date = [
                    t.get('Value') for t in image.tags
                    if t['Key'] == 'backup'
                ][0]
                imagesList.append(image.id)
                
                
       except IndexError:
           deletion_date = False
           delete_date = False

       today_time = datetime.datetime.now().strftime('%m-%d-%Y')
        # today_fmt = today_time.strftime('%m-%d-%Y')
       today_date = time.strptime(today_time, '%m-%d-%Y')

                     
       
print ("=============")

print ("About to process the following AMIs:")
print ('imagesList')

myAccount = boto3.client('sts').get_caller_identity()['Account']

snapshots = ec.describe_snapshots(MaxResults=1000, OwnerIds=[myAccount])['Snapshots']

# loop through list of image IDs
for image in imagesList:
    try:
        print ("deregistering image %s" % image)
        amiResponse = ec.deregister_image(
        DryRun=False,
        ImageId=image,
       )
    except ClientError:
        print ("ClientError, likely AMI not found")
        continue

    for snapshot in snapshots:
        if snapshot['Description'].find(image) > 0:
            snap = ec.delete_snapshot(SnapshotId=snapshot['SnapshotId'])
            print ("Deleting snapshot " + snapshot['SnapshotId'])

print ("-------------")
+++++++++++++++

How to tag snapshot by using ami id created by this script

Thank you for the nice script. :)

Hi thanks for this nice script. I have made a small enhancement on the script, so it also supports instances with multiple EBS volumes attached.

On line 42 i have added a new empty list, that will hold all instances that where already done :
inDone = []

And the line 67 should be surrounded by a check. Just replace the line 67 with:

            if str(instance['InstanceId']) not in inDone:

              AMIid = ec.create_image(InstanceId=instance['InstanceId'], Name="Lambda - " + instance['InstanceId'] + " from " + create_fmt, Description="Lambda created AMI of instance " + instance['InstanceId'] + " from " + create_fmt, NoReboot=True, DryRun=False)

              inDone.insert(0,str(instance['InstanceId']))

              print "Created AMI %s of instance %s " % (AMIid['ImageId'], instance['InstanceId'])

            else:

              print "We already got an AMI of instance %s " % (instance['InstanceId'])

Thanks again!
Regards, Frank

Hello Frank,

Can you paste me the full script along with modifications you made