Scale your aws Fargate task 0 to 100 and 100 to 0

readme.md

Scaling up and down AWS Fargate with Lambda and SQS

Goal

Set up Lambda, which reads sqs-queue every minute and decides to start or stop aws-fargate tasks.

Literature

• https://github.com/miketheman/ecs-host-service-scale

• https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sqs.html#SQS.Queue.attributes -> ApproximateNumberOfMessages

  import boto.sqs
  c = boto.sqs.connect_to_region('us-east-1')
  queue = c.lookup('myqueue')
  attr = queue.get_attributes() 

• https://lobster1234.github.io/2017/12/03/run-tasks-with-aws-fargate-and-lambda/

• https://stackoverflow.com/questions/53860449/schedule-aws-ecs-fargate-to-start-and-stop-tasks

• https://docs.aws.amazon.com/de_de/AWSCloudFormation/latest/UserGuide/aws-resource-ecs-service.html

• pipeline https://docs.aws.amazon.com/de_de/AmazonECS/latest/developerguide/ecs-cd-pipeline.html

procedure

• Use Serverless Framework for lambda function
• use Cloudformation for ECR deploy and Fargate Definition -> show docs
• script for docker build and deploy

handler.py

import boto3
import os

client = boto3.client('ecs', region_name='eu-central-1')

def autoscaler(event,context):
    message_number = count_sqs(queue_url)
    desired_count = evaluate_scale(message_number)
        
    response = client.update_service(
        cluster=os.environ['test_ecs_cluster'],
        service=os.environ['test_ecs_service'],
        taskDefinition=os.environ['test_ecs_task'],
        desiredCount=desired_count
    )
    return true
# Required env vars:
# $test_ecs_cluster: name of the ECS cluster
# $test_ecs_task: name and revision of the task definition (i.e. `mytask:1`)
# $test_ecs_service: name of the ECS service