Skip to content

Instantly share code, notes, and snippets.

@darkarnium
Last active September 26, 2022 08:16
Show Gist options
  • Save darkarnium/d8e6be95d582a30031b646c41f96e139 to your computer and use it in GitHub Desktop.
Save darkarnium/d8e6be95d582a30031b646c41f96e139 to your computer and use it in GitHub Desktop.
A quick and dirty AWS EC2 Spot Fleet requestor

SpotFleets

The following code will request an AWS SpotFleet with the specified parameters.

Credentials

By default credentials will be located using boto3's built-in enumeration mechanism. The easiest way to ensure that credentials are available is to either use environment variables, or ensure there is a ~/.aws/credentials file for the user running this script.

Required Code Changes

The code needs to be updated and the SpotFleetRequestConfig updated with:

  • Aa IamFleetRole ARN (the default aws-ec2-spot-fleet-role user can be used here).
  • An ImageId to use for created instances (AMI).
  • A KeyName to use for created instances (SSH Key).
  • An array of SecurityGroup hashes containing the GroupId to use for created instances.
  • A SubnetId to use for created instances.

Usage

Usage: spotInstance.py [OPTIONS]

  Spot Instance Scheduler.

  This code provides a simple interface to the Amazon Web Services (AWS) EC2
  Spot Fleet API. It allows for a fleet of EC2 nodes to be provisioned if
  the price falls between the desired figures.

Options:
  --client-token TEXT     A unique identifier to prevent duplicate fleets
  --instance-type TEXT    Type of instance to request.
  --target-region TEXT    The region to request instances in.
  --target-count INTEGER  The target number of instances.
  --target-price TEXT     The target bid price, per hour.
  --help                  Show this message and exit.

Example

python spotInstance.py --instance-type 'm3.medium' --target-region 'us-west-2' --target-count 2 --target-price '0.10'
2017-04-25 22:57:09,420 - 8144 - [INFO] A unique fleet token of X is being used for this deployment
2017-04-25 22:57:09,427 - 8144 - [INFO] Found credentials in shared credentials file: ~/.aws/credentials
2017-04-25 22:57:09,528 - 8144 - [INFO] Starting new HTTPS connection (1): ec2.us-east-1.amazonaws.com
2017-04-25 22:57:10,607 - 8144 - [INFO] Requesting fleet between 2017-04-26 05:57:10.607000 and 2017-05-03 05:57:10.607000 (7 days)
2017-04-25 22:57:10,608 - 8144 - [INFO] Attempting to create a fleet of 2x m3.medium at a price of $0.10 in us-west-2
2017-04-25 22:57:10,611 - 8144 - [INFO] Starting new HTTPS connection (1): ec2.us-west-2.amazonaws.com
2017-04-25 22:57:11,319 - 8144 - [INFO] Spot fleet requested! Reference is sfr-cd508177-3d63-4d98-933c-139cd9c089f9

Code

"""Spot Instance Scheduler.

This code provides a simple interface to the Amazon Web Services (AWS) EC2
Spot Fleet API. It allows for a fleet of EC2 nodes to be provisioned if the
price falls between the desired figures.
"""
import sys
import logging
import datetime
import boto3
import botocore
import click


@click.command()
@click.option('--client-token', help='A unique identifier to prevent duplicate fleets', default='X')
@click.option('--instance-type', help='Type of instance to request.')
@click.option('--target-region', help='The region to request instances in.', default='us-west-2')
@click.option('--target-count', help='The target number of instances.', type=int)
@click.option('--target-price', help='The target bid price, per hour.')
def main(client_token, instance_type, target_region, target_count, target_price):
    """Spot Instance Scheduler.

    This code provides a simple interface to the Amazon Web Services (AWS) EC2
    Spot Fleet API. It allows for a fleet of EC2 nodes to be provisioned if the
    price falls between the desired figures.
    """
    logging.basicConfig(
        level=logging.INFO,
        format='%(asctime)s - %(process)d - [%(levelname)s] %(message)s',
    )
    logger = logging.getLogger()
    logger.info('A unique fleet token of %s is being used for this deployment', client_token)

    # Ensure requires parameters are set.
    if instance_type is None:
        logger.fatal('No instance-type provided, cannot continue.')
        sys.exit(-1)
    if target_count is None:
        logger.fatal('No target instance count provided, cannot continue.')
        sys.exit(-1)
    if target_price is None:
        logger.fatal('No target price provided, cannot continue.')
        sys.exit(-1)

    # Ensure the requested region exists.
    ec2 = boto3.client('ec2')
    regions = ec2.describe_regions()
    for region in regions['Regions']:
        if region['RegionName'].lower() == target_region.lower():
            break

    if not region:
        logger.fatal('Could not find a region named %s, cannot continue', target_region)
        sys.exit(-2)

    # Connect to EC2.
    ec2 = boto3.client('ec2', region_name=region['RegionName'])

    # Define dates.
    date_from = datetime.datetime.utcnow()
    date_to = date_from + datetime.timedelta(days=7)
    logger.info('Requesting fleet between %s and %s (%d days)', date_from, date_to, 7)

    # Create the fleet.
    logger.info(
        'Attempting to create a fleet of %dx %s at a price of $%s in %s',
        target_count,
        instance_type,
        target_price,
        target_region
    )
    try:
        request = ec2.request_spot_fleet(
            SpotFleetRequestConfig={
                'ClientToken': client_token,
                'SpotPrice': target_price,
                'TargetCapacity': target_count,
                # Uncomment the following 'lease' from now to 7-days from now.
                # 'ValidFrom': date_from.strftime('%Y-%m-%dT%H:%M:%SZ'),
                # 'ValidUntil': date_to.strftime('%Y-%m-%dT%H:%M:%SZ'),
                'IamFleetRole': 'arn:aws:iam::WWWWWWWWWWWW:role/aws-ec2-spot-fleet-role',
                'LaunchSpecifications': [
                    {
                        'ImageId': 'ami-XXXXXXXX',
                        'KeyName': 'CHANGEME',
                        'SecurityGroups': [
                            {
                                'GroupId': 'sg-YYYYYYYY'
                            }
                        ],
                        'UserData': '',
                        'InstanceType': instance_type,
                        'SubnetId': 'subnet-ZZZZZZZZ'
                    }
                ],
                'AllocationStrategy': 'lowestPrice'
            }
        )
    except botocore.exceptions.ParamValidationError as err:
        logger.fatal('Bad parameters provided, cannot continue: %s', err)
        sys.exit(-3)
    except botocore.exceptions.ClientError as err:
        logger.fatal('Failed to request spot fleet, cannot continue: %s', err)
        sys.exit(-4)

    # All done!
    logger.info('Spot fleet requested! Reference is %s', request['SpotFleetRequestId'])

if __name__ == '__main__':
    main()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment