AWS EC2 Unused Security Group cleanup

security-group-cleanup.py

#!/usr/bin/env python

import sys
import boto
import pprint

del_flag = ''
if len(sys.argv) > 1:
    del_flag = sys.argv[1]

pp = pprint.PrettyPrinter(indent=4)

# set credentials
ACCESS_KEY="<access key>"
SECRET_KEY="<security key>"

ec2 = boto.connect_ec2(ACCESS_KEY, SECRET_KEY)

allgroups = []
# Get ALL security groups names
groups = ec2.get_all_security_groups()
for groupobj in groups:
    allgroups.append(groupobj.name)
# pp.pprint(sorted(allgroups))

# Get [running|stopped] instances security groups
groups_in_use = []
for state in ['running','stopped']:
    reservations = ec2.get_all_instances(filters={'instance-state-name': state})
    for r in reservations:
        for inst in r.instances:
            if inst.groups[0].name not in groups_in_use:
                groups_in_use.append(inst.groups[0].name)

delete_candidates = []
for group in allgroups:
    if group not in groups_in_use:
        delete_candidates.append(group)

if del_flag == '--delete':
    print "We will now delete security groups identified to not be in use."
    for group in delete_candidates:
        ec2.delete_security_group(group)
    print "We have deleted %d groups." % (len(delete_candidates))
else:
    print "The list of security groups to be removed is below."
    print "Run this again with `--delete` to remove them"
    pp.pprint(sorted(delete_candidates))
    print "Total of %d groups targeted for removal." % (len(delete_candidates))




# For each security group in the total list, if not in the "used" list, flag for deletion
# If running with a "--delete" flag, delete the ones flagged.

I forked @irontoby's — and believe I fixed another bug:

There's this assumption that an instance has one security group in the code above. I've fixed this in my fork. Otherwise, you run this code, it may attempt to delete security groups which are in fact used by instances.

I added a fork because I was still getting a list of security groups with the instances that had multiple security groups. Now I think that is fixed. added a report at the end of what it evaluated. Changed it to be for every ec2 instance not just specific states.

Thanks a lot, @dritten.

I can only see default group , believe this is becasue region is set to us-east by default, how can this be changed to west ? I've tried boto.ec2.connect_to_region("us-west-2a", ACCESS_KEY, SECRET_KEY) and importing boto.ec2

@deeco you can just replace:

ec2 = boto.connect_ec2(ACCESS_KEY, SECRET_KEY)

with:

ec2 = boto.connect_ec2_endpoint(url='http://<your_aws_endpoint/',aws_access_key_id=ACCESS_KEY, aws_secret_access_key=SECRET_KEY)

also, you might want to hold back on using the delete argument as it might try as the script is listing load balancer security groups as not being used and therefore candidates for deletion.

another version for the same thing

https://github.com/codingchef/scripts/blob/master/python/findsg.py

I created a fork of dritten's version of this; I converted it to boto3/python 3.5. I also included VPCs, since they now claim security groups, and I've added an exception to note security groups that could not be automatically deleted.

I've got a fork based on a sa-jbrooks and a number of other forks. Checks ELBs/RDS/NetworkInterfaces, also checks for OpsWorks / Directory Service SG's.

@TomRyan-321: Link please?

EDIT: This one? https://gist.github.com/TomRyan-321/0cf6e48937cbe9513afc50117d6ffd6f

This just checks if a Security Group is attached to any EC2 instances. Doesn't tell you if the group is being used by an RDS instance (launched by aws on your behalf, not returned by get_all_instances) or ELB.

There is a tool (disclaimer : I worked on it) that identifies detached Security Groups, but also matches up your VPC Flow Logs against the Security Group rules, to tell you which individual rules are used and which are unused, making cleanup more thorough. Here's a little more information if anyone is curious: https://www.piasoftware.net/single-post/2018/04/24/VIDEO-Watch-as-we-clean-up-EC2-security-groups-in-just-a-few-minutes

I am getting below when executing with --delete argument, boto.exception.EC2ResponseError: EC2ResponseError: 400 Bad Request

InvalidGroup.NotFoundThe security group 'launch-wizard-15' does not exist in default VPC

Also, sometimes SGs are references as scheduled tasks in ECS, you might delete a SG that is referenced there.