Connect Fargate instance to SSM Session Manager

docker-entrypoint.sh

#!/bin/bash

INSTANCE_NAME=acme-development
AWS_REGION="$(aws configure get region)"

################################################################################
# The section below obtains an activation code and ID from SSM, and then uses it
# to register the current agent. _This should only be done on the basis of
# tightly controlled roles granted to ECS._ Note that it is registered with two 
# tags:
#
#     Name:    While the name is set via --default-instance-name, the name will
#              only show up when queries are performed in the CLI. The "Name"
#              tag is required for the name to be visible in the AWS console.
#     Type:    This acts a flag, so that only offline Fargate instances get 
#              cleaned up.
#
# The SSM agent is then started. Output is redirected to STDOUT and the process
# is sent to the background. Both of these actions are require to prevent the
# agent from blocking the script.
################################################################################
read -r ACTIVATION_CODE ACTIVATION_ID <<< $(aws ssm create-activation --default-instance-name "${INSTANCE_NAME}" --iam-role "SSMServiceRole" --registration-limit 1 --tags "Key=Name,Value=${INSTANCE_NAME}" "Key=Type,Value=fargate" --query "join(' ', [ActivationCode, ActivationId])" --output text)
amazon-ssm-agent -register -code "${ACTIVATION_CODE}" -id "${ACTIVATION_ID}" -region "${AWS_REGION}" -clear -y
amazon-ssm-agent >&1 &

# Manage the logs by redirecting output to CloudWatch log groups...

Dockerfile

FROM debian:10-slim

RUN apt-get update -y && \
  apt-get install -y awscli curl gnupg && \
  apt-key adv --fetch-keys "https://nginx.org/keys/nginx_signing.key" && \
  echo "deb http://nginx.org/packages/debian buster nginx" > /etc/apt/sources.list.d/nginx.list

RUN curl --silent --show-error --location --output /tmp/amazon-ssm-agent.deb "https://s3.us-east-1.amazonaws.com/amazon-ssm-us-east-1/latest/debian_amd64/amazon-ssm-agent.deb" && \
  dpkg -i /tmp/amazon-ssm-agent.deb

COPY docker-entrypoint.sh /

EXPOSE 80
ENTRYPOINT [ "/docker-entrypoint.sh" ]
CMD [ "nginx" ]

index.js

const { SSM } = require('aws-sdk');
const ssm = new SSM();

exports.handler = async (event, context, callback) => {
    const { InstanceInformationList } = await ssm.describeInstanceInformation({
        Filters: [
            { Key: 'tag:Type', Values: [ 'fargate' ] },
        ]
    }).promise();
    
    const offlineInstanceIds = InstanceInformationList.reduce(( accumulator, { InstanceId, PingStatus } ) => {
        if(PingStatus != 'Online') accumulator.push(InstanceId);
        
        return accumulator;
    }, []);
    
    const totalOfflineInstances = offlineInstanceIds.length;
    const deregisteredOfflineInstances = 0;
    
    for (var offlineInstanceId of offlineInstanceIds) {
        try {
            await ssm.deregisterManagedInstance({ InstanceId: offlineInstanceId }).promise()
            deregisteredOfflineInstances++;
        } catch (e) {}
    }
    
    console.log(`Deregistered ${ deregisteredOfflineInstances } of ${ totalOfflineInstances } offline Fargate instances`);
    callback(null);
};

Disclaimer: I know there are better ways to run services in Docker... This will get more polished as it settles into my workflow.

How does this work?

What this is going to do is set up a managed instance in a hybrid environment... What does that mean in non-AWS English? AWS will let you manage on-premises servers/containers via SSM. All this does is register your Fargate instance with SSM as if it were an on-premises managed server.

In Dockerfile we download and install the SSM agent. This happens in the build phase, so we don't need to worry about stopping it.

In docker-entrypoint.sh we first request an activation, so that we can attach our "on-premises" server (i.e. Fargate instance) to SSM. An activation code and ID are provided with a default lifespan of 24 hours. That means you have 24 hours to register that instance with SSM, which is what we do next. Finally, we run the agent. If you've followed the steps below, you should be on your way to session managing your Fargate instances!

The index.js file is intended to be a quick 'n nasty Lambda function for cleaning up left-over Fargate managed instances. It lists all the instances that are tagged with "fargate", then filters out any that are actually online. Then, it deregisters whatever it has found, hopefully mitigating 💸.

What do you need to do to get this running?

Here is a bit of to-do list you'll need to consider before this is going to start working:

• You will need to activate Advanced Instances in SSM. At the time of writing, each advanced instance costs ~$5 per month if they are left running 24/7.
• You will need to create a service role for SSM that allows to grab tokens from STS for authentication.
• You will need to create an create a service role for ECS that allows it to activate and register instances.
• You will need to create a Lambda function and dump the contents of index.js into it. You will also need to create an IAM role that allows you describe and de-register instances. (The specific actions to allow this aressm:DeregisterManagedInstance and ssm:DescribeInstanceInformation) Finally, create a scheduled CloudWatch event rule that calls your function at whatever duration you feel is neccessary.

What do I need to do to make this better?

• Add some graceful error handling
• Reduce activation token lifespan because we register it almost instantly
• Handle the service better as it's likely going to be running alongside other critical services
• Send logs to CloudWatch, which will require adding CloudWatchAgentServerPolicy to the service role for SSM, and some finagling with Seelog configs
• Remove registered instances (possibly with SNS and Lambda?) when instances stop
• Implement Lambda cleanup and manual advanced instance activation and service role creation with a little AWS CDK script and dump this into a proper repo

awesome work above!! AWS must have read this and taken note!

https://aws.amazon.com/blogs/containers/new-using-amazon-ecs-exec-access-your-containers-fargate-ec2/