Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Workaround for 'awsvpc' network-mode containers and multiple target groups from https://github.com/aws/amazon-ecs-agent/issues/1351
AWSTemplateFormatVersion: 2010-09-09
Description: |
Replicates (de)registration events from a source target group to a destionation target group
in order to workaround ECS limitation where a single container cannot automatically register
to multiple target groups
Parameters:
SourceTargetGroupArn:
Description: ARN of a source target group to replicate targets from
Type: String
DestinationTargetGroupArn:
Description: ARN of target group to replicate targets to
Type: String
Outputs:
AutoUpdateTargetGroupLambdaName:
Value: !Ref AutoUpdateTargetGroupLambda
AutoUpdateTargetGroupLambdaArn:
Value: !GetAtt AutoUpdateTargetGroupLambda.Arn
Resources:
TargetGroupRegistrationEvents:
Type: AWS::Events::Rule
Properties:
Description: Watch for (De)Register events from source target group
State: ENABLED
EventPattern:
source:
- aws.elasticloadbalancing
detail-type:
- AWS API Call via CloudTrail
detail:
eventSource:
- elasticloadbalancing.amazonaws.com
eventName:
- RegisterTargets
- DeregisterTargets
requestParameters:
targetGroupArn:
- !Ref SourceTargetGroupArn
Targets:
- Arn: !GetAtt AutoUpdateTargetGroupLambda.Arn
Id: lambda
AutoUpdateTargetLambdaInvokePermission:
Type: AWS::Lambda::Permission
Properties:
Action: "lambda:InvokeFunction"
FunctionName: !GetAtt AutoUpdateTargetGroupLambda.Arn
Principal: events.amazonaws.com
SourceArn: !GetAtt TargetGroupRegistrationEvents.Arn
AutoUpdateTargetGroupLambdaRole:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Principal:
Service: lambda.amazonaws.com
Action: "sts:AssumeRole"
ManagedPolicyArns:
- "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
Policies:
- PolicyName: AutoUpdateTargetGroupMembers
PolicyDocument:
Version: 2012-10-17
Statement:
- Effect: Allow
Action:
- "elasticloadbalancing:RegisterTargets"
- "elasticloadbalancing:DeregisterTargets"
Resource: !Ref DestinationTargetGroupArn
Path: /service-role/
AutoUpdateTargetGroupLambda:
Type: AWS::Lambda::Function
Properties:
Description: Automatically (De)Registers destination target group(s) when a source target group changes
Environment:
Variables:
DEST_ARN: !Ref DestinationTargetGroupArn
Handler: index.lambda_handler
MemorySize: 128
Role: !GetAtt AutoUpdateTargetGroupLambdaRole.Arn
Runtime: python3.6
Timeout: 10
Code:
ZipFile: |
import boto3
import logging
import os
DEST_ARN = os.environ["DEST_ARN"]
def lambda_handler(event, context):
"""
Given the CloudWatch events 'RegisterTargets' or 'DeregisterTargets',
replay (almost) the same events on another target group
"""
# Initialize logger
logger = logging.getLogger()
logger.setLevel(logging.INFO)
logger.debug(os.environ)
logger.debug(event)
logger.debug(context)
# Instantiate API client for load balancers
client = boto3.client("elbv2")
# Gather relevant info from CloudWatch event
event_name = event["detail"]["eventName"]
event_targets = event["detail"]["requestParameters"]["targets"]
# Log the relevant information from the received event
logger.info("Received CloudWatch Event '%s' with the following targets:" % event_name)
logger.info(event_targets)
# Reformat / generalize targets
targets = list(map(
lambda target: {"Id": target["id"]},
event_targets
))
# Register new targets
if event_name == "RegisterTargets":
response = client.register_targets(
TargetGroupArn=DEST_ARN,
Targets=targets
)
# Or Deregister old targets
elif event_name == "DeregisterTargets":
response = client.deregister_targets(
TargetGroupArn=DEST_ARN,
Targets=targets
)
# Raise an exception if the Lambda received an unanticipated event
else:
raise ValueError("Unrecognized event type")
return {
"statusCode": 200,
"body": {
"event_processed": event_name,
"response": response
}
}
@arpaulnet
Copy link
Author

arpaulnet commented Jun 28, 2019

This was only tested with awsvpc ECS tasks and ip (not instance) target groups. It works by capturing the register / deregister events from one target group then registering the same target on another target group which has a different port already configured. The cloudformation template will not bring the target groups up-to-date on instantiation, so either manually register the initial targets or re-register the targets by temporarily killing the appropriate service tasks.

@arpaulnet
Copy link
Author

arpaulnet commented Aug 1, 2019

For any who find this gist in the future, as of July 30 2019 AWS ECS now has this feature built-in: https://aws.amazon.com/about-aws/whats-new/2019/07/amazon-ecs-services-now-support-multiple-load-balancer-target-groups/ .

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment