ChatGPT: Build a script with ChatGPT to delete excess Bento Remote AMIs

ChatGPT4.md

👱 detect and remove unused AWS AMIs

🤖 As a helpful assistant, I can provide you with the steps to detect and remove unused Amazon Machine Images (AMIs) in your AWS account. However, I cannot perform the actions for you. Follow these steps to detect and remove unused AMIs:

1. Install and configure AWS CLI: If you haven't already, install the AWS Command Line Interface (CLI) and configure it with your AWS account credentials. You can find the instructions in the AWS documentation: Installing, updating, and uninstalling the AWS CLI and Configuration basics.

2. List all AMIs: Use the following command to list all the AMIs owned by your AWS account:

   aws ec2 describe-images --owners self
   

   This command will return a JSON object containing information about the AMIs. You can filter the output to show only the AMI IDs using the --query option:

   aws ec2 describe-images --owners self --query 'Images[*].{ID:ImageId}' --output text
   

3. Identify unused AMIs: To detect unused AMIs, you can list all the instances in your account and check which AMIs are being used. Run the following command to list all instances:

   aws ec2 describe-instances --query 'Reservations[*].Instances[*].{ID:InstanceId,AMI:ImageId}' --output text
   

   Compare the list of AMIs used by instances with the list of all AMIs you own. Any AMI not used by an instance can be considered unused.

4. Deregister unused AMIs: To remove an unused AMI, you need to deregister it. Use the following command to deregister an AMI, replacing ami-xxxxxxxxxxxxxxxxx with the AMI ID:

   aws ec2 deregister-image --image-id ami-xxxxxxxxxxxxxxxxx
   

   Warning: Deregistering an AMI will make it unavailable for use in the future. Ensure that the AMI is not needed before deregister

👱 Show me how to delete all AMIs older than two weeks and tagged with role=bento-remote

🤖 To delete all AMIs older than two weeks and tagged with role=bento-remote, you can follow these steps:

1. Install and configure AWS CLI: If you haven't already, install the AWS Command Line Interface (CLI) and configure it with your AWS account credentials. You can find the instructions in the AWS documentation: Installing, updating, and uninstalling the AWS CLI and Configuration basics.

2. Find AMIs with the specified tag: Use the following command to list all the AMIs with the tag role=bento-remote:

   aws ec2 describe-images --owners self --filters "Name=tag:role,Values=bento-remote"
   

3. Filter AMIs older than two weeks: You can use a script to filter the AMIs older than two weeks. Here's a Python script that does this:

   import json
   import sys
   from datetime import datetime, timedelta
   
   ami_data = json.load(sys.stdin)
   two_weeks_ago = datetime.now() - timedelta(weeks=2)
   
   for ami in ami_data["Images"]:
       creation_date = datetime.strptime(ami["CreationDate"], "%Y-%m-%dT%H:%M:%S.%fZ")
       if creation_date < two_weeks_ago:
           print(ami["ImageId"])

   Save this script to a file, e.g., filter_old_amis.py. Then, run the following command to get the list of AMIs older than two weeks:

   aws ec2 describe-images --owners self --filters "Name=tag:role,Values=bento-remote" | python filter_old_amis.py
   

4. Deregister and delete AMIs: Use the following shell script to deregister the AMIs and delete the associated snapshots:

   #!/bin/bash
   
   OLD_AMIS=$(aws ec2 describe-images --owners self --filters "Name=tag:role,Values=bento-remote" | python filter_old_amis.py)
   
   for ami_id in $OLD_AMIS; do
       echo "Deregistering AMI: $ami_id"
       aws ec2 deregister-image --image-id "$ami_id"
   
       snapshot_id=$(aws ec2 describe-images --image-ids "$ami_id" --query 'Images[*].BlockDeviceMappings[*].Ebs.SnapshotId' --output text)
       if [ -n "$snapshot_id" ]; then
           echo "Deleting snapshot: $snapshot_id"
           aws ec2 delete-snapshot --snapshot-id "$snapshot_id"
       fi
   done

   Save this script to a file, e.g., delete_old_amis.sh, and make it executable by running chmod +x delete_old_amis.sh. Then, execute the script:

   ./delete_old_amis.sh
   

This script will deregister all AMIs older than two weeks and tagged with role=bento-remote, and delete the associated snapshots. Ensure that you have a backup or snapshot of the AMIs before running the script, as this action is irreversible.

👱 now do all of this in one python script

🤖 Here's a single Python script that combines all the steps to delete AMIs older than two weeks and tagged with role=bento-remote:

import boto3
from datetime import datetime, timedelta

# Initialize the EC2 client
ec2 = boto3.client('ec2')

# Get the current date and two weeks ago
current_date = datetime.now()
two_weeks_ago = current_date - timedelta(weeks=2)

# Get the list of AMIs with the specified tag
response = ec2.describe_images(Owners=['self'], Filters=[{'Name': 'tag:role', 'Values': ['bento-remote']}])

# Iterate through the list and delete the AMIs older than two weeks
for ami in response['Images']:
    ami_id = ami['ImageId']
    creation_date = datetime.strptime(ami['CreationDate'], "%Y-%m-%dT%H:%M:%S.%fZ")

    if creation_date < two_weeks_ago:
        print(f'Deregistering AMI: {ami_id}')
        ec2.deregister_image(ImageId=ami_id)

        for block_device in ami['BlockDeviceMappings']:
            if 'Ebs' in block_device:
                snapshot_id = block_device['Ebs']['SnapshotId']
                print(f'Deleting snapshot: {snapshot_id}')
                ec2.delete_snapshot(SnapshotId=snapshot_id)

Save this script as delete_old_amis.py and make sure you have the boto3 library installed. If you don't have it installed, you can install it using pip:

pip install boto3

After installing boto3, ensure that your AWS credentials are properly configured. You can either set the environment variables AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_DEFAULT_REGION, or configure the credentials using the aws configure command.

Finally, run the script:

python delete_old_amis.py

This script will deregister all AMIs older than two weeks and tagged with role=bento-remote, and delete the associated snapshots. Ensure that you have a backup or snapshot of the AMIs before running the script, as this action is irreversible.

👱 Can you modify just this line to add another filter so that AMI has both tags role=bento-remote and bento_image_type=profile

🤖 Certainly! You can add another filter to the describe_images function to include the bento_image_type=profile tag. Here's the modified line:

response = ec2.describe_images(Owners=['self'], Filters=[{'Name': 'tag:role', 'Values': ['bento-remote']}, {'Name': 'tag:bento_image_type', 'Values': ['profile']}])

With this change, the script will now only process AMIs that have both role=bento-remote and bento_image_type=profile tags.

👱 Can you print out the AMI name before deregistering it? Just show the code please

🤖 Certainly! You can add a line to print the AMI name before deregistering it. Here's the modified code snippet:

# Iterate through the list and delete the AMIs older than two weeks
for ami in response['Images']:
    ami_id = ami['ImageId']
    ami_name = ami['Name']
    creation_date = datetime.strptime(ami['CreationDate'], "%Y-%m-%dT%H:%M:%S.%fZ")

    if creation_date < two_weeks_ago:
        print(f'AMI Name: {ami_name}')
        print(f'Deregistering AMI: {ami_id}')
        ec2.deregister_image(ImageId=ami_id)

        for block_device in ami['BlockDeviceMappings']:
            if 'Ebs' in block_device:
                snapshot_id = block_device['Ebs']['SnapshotId']
                print(f'Deleting snapshot: {snapshot_id}')
                ec2.delete_snapshot(SnapshotId=snapshot_id)

Now, the script will print the AMI name before deregistering it.

👱 Now modify this to keep the newest 10 AMIs bucketed by a tag called profile

🤖 To keep the newest 10 AMIs for each profile, you can modify the script as follows:

import boto3
from datetime import datetime
from collections import defaultdict

# Initialize the EC2 client
ec2 = boto3.client('ec2')

# Get the list of AMIs with the specified tag
response = ec2.describe_images(Owners=['self'], Filters=[{'Name': 'tag:role', 'Values': ['bento-remote']}, {'Name': 'tag:bento_image_type', 'Values': ['profile']}])

# Group the AMIs by profile tag
amis_by_profile = defaultdict(list)
for ami in response['Images']:
    profile = None
    for tag in ami['Tags']:
        if tag['Key'] == 'profile':
            profile = tag['Value']
            break

    if profile:
        amis_by_profile[profile].append(ami)

# Sort the AMIs by creation date and keep the newest 10 for each profile
for profile, amis in amis_by_profile.items():
    sorted_amis = sorted(amis, key=lambda x: x['CreationDate'], reverse=True)
    amis_to_delete = sorted_amis[10:]

    # Delete the older AMIs
    for ami in amis_to_delete:
        ami_id = ami['ImageId']
        ami_name = ami['Name']
        print(f'AMI Name: {ami_name}')
        print(f'Deregistering AMI: {ami_id}')
        ec2.deregister_image(ImageId=ami_id)

        for block_device in ami['BlockDeviceMappings']:
            if 'Ebs' in block_device:
                snapshot_id = block_device['Ebs']['SnapshotId']
                print(f'Deleting snapshot: {snapshot_id}')
                ec2.delete_snapshot(SnapshotId=snapshot_id)

This script will group the AMIs by the profile tag, sort them by creation date, and keep only the newest 10 AMIs for each profile. It will deregister and delete the snapshots of the older AMIs.

👱 can you delete 10 of them in parallel?

🤖 To delete the AMIs in parallel, you can use Python's concurrent.futures library with a thread pool. Here's the modified script that deletes 10 AMIs in parallel:

import boto3
from datetime import datetime
from collections import defaultdict
import concurrent.futures

# Function to deregister an AMI and delete its snapshot
def delete_ami(ami):
    ami_id = ami['ImageId']
    ami_name = ami['Name']
    print(f'AMI Name: {ami_name}')
    print(f'Deregistering AMI: {ami_id}')
    ec2.deregister_image(ImageId=ami_id)

    for block_device in ami['BlockDeviceMappings']:
        if 'Ebs' in block_device:
            snapshot_id = block_device['Ebs']['SnapshotId']
            print(f'Deleting snapshot: {snapshot_id}')
            ec2.delete_snapshot(SnapshotId=snapshot_id)

# Initialize the EC2 client
ec2 = boto3.client('ec2')

# Get the list of AMIs with the specified tag
response = ec2.describe_images(Owners=['self'], Filters=[{'Name': 'tag:role', 'Values': ['bento-remote']}, {'Name': 'tag:bento_image_type', 'Values': ['profile']}])

# Group the AMIs by profile tag
amis_by_profile = defaultdict(list)
for ami in response['Images']:
    profile = None
    for tag in ami['Tags']:
        if tag['Key'] == 'profile':
            profile = tag['Value']
            break

    if profile:
        amis_by_profile[profile].append(ami)

# Sort the AMIs by creation date and keep the newest 10 for each profile
for profile, amis in amis_by_profile.items():
    sorted_amis = sorted(amis, key=lambda x: x['CreationDate'], reverse=True)
    amis_to_delete = sorted_amis[10:]

    # Delete the older AMIs in parallel
    with concurrent.futures.ThreadPoolExecutor() as executor:
        executor.map(delete_ami, amis_to_delete)

This script uses a thread pool to delete the older AMIs in parallel, which should speed up the deletion process. Keep in mind that the number of threads used by the ThreadPoolExecutor defaults to the number of processors on the machine. If you want to limit the number of threads, you can pass the max_workers parameter to the ThreadPoolExecutor, like this:

with concurrent.futures.ThreadPoolExecutor(max_workers=10) as executor:
    executor.map(delete_ami, amis_to_delete)

👱 now print out the progress every 100 you delete

🤖 To print the progress every 100 AMIs deleted, you can modify the delete_ami function to accept an index and a shared counter. Here's the updated script that prints the progress:

import boto3
from datetime import datetime
from collections import defaultdict
import concurrent.futures
from threading import Lock

# Function to deregister an AMI and delete its snapshot
def delete_ami(ami, index, counter, lock):
    ami_id = ami['ImageId']
    ami_name = ami['Name']
    print(f'AMI Name: {ami_name}')
    print(f'Deregistering AMI: {ami_id}')
    ec2.deregister_image(ImageId=ami_id)

    for block_device in ami['BlockDeviceMappings']:
        if 'Ebs' in block_device:
            snapshot_id = block_device['Ebs']['SnapshotId']
            print(f'Deleting snapshot: {snapshot_id}')
            ec2.delete_snapshot(SnapshotId=snapshot_id)

    with lock:
        counter[0] += 1
        if counter[0] % 100 == 0:
            print(f'Progress: {counter[0]} AMIs deleted')

# Initialize the EC2 client
ec2 = boto3.client('ec2')

# Get the list of AMIs with the specified tag
response = ec2.describe_images(Owners=['self'], Filters=[{'Name': 'tag:role', 'Values': ['bento-remote']}, {'Name': 'tag:bento_image_type', 'Values': ['profile']}])

# Group the AMIs by profile tag
amis_by_profile = defaultdict(list)
for ami in response['Images']:
    profile = None
    for tag in ami['Tags']:
        if tag['Key'] == 'profile':
            profile = tag['Value']
            break

    if profile:
        amis_by_profile[profile].append(ami)

# Sort the AMIs by creation date and keep the newest 10 for each profile
for profile, amis in amis_by_profile.items():
    sorted_amis = sorted(amis, key=lambda x: x['CreationDate'], reverse=True)
    amis_to_delete = sorted_amis[10:]

    # Delete the older AMIs in parallel
    counter = [0]
    lock = Lock()
    with concurrent.futures.ThreadPoolExecutor() as executor:
        futures = [executor.submit(delete_ami, ami, index, counter, lock) for index, ami in enumerate(amis_to_delete)]
        concurrent.futures.wait(futures)

This script uses a shared counter and a lock to keep track of the number of deleted AMIs. It prints the progress every 100 AMIs deleted.