Workaround for 'awsvpc' network-mode containers and multiple target groups from https://github.com/aws/amazon-ecs-agent/issues/1351

awsvpc-mode-multi-targetgroup-alb.yml

AWSTemplateFormatVersion: 2010-09-09

Description: | 
  Replicates (de)registration events from a source target group to a destionation target group
  in order to workaround ECS limitation where a single container cannot automatically register
  to multiple target groups

Parameters:
  SourceTargetGroupArn:
    Description: ARN of a source target group to replicate targets from
    Type: String
  DestinationTargetGroupArn:
    Description: ARN of target group to replicate targets to
    Type: String

Outputs:
  AutoUpdateTargetGroupLambdaName:
    Value: !Ref AutoUpdateTargetGroupLambda
  AutoUpdateTargetGroupLambdaArn:
    Value: !GetAtt AutoUpdateTargetGroupLambda.Arn

Resources:
  TargetGroupRegistrationEvents:
    Type: AWS::Events::Rule
    Properties:
      Description: Watch for (De)Register events from source target group
      State: ENABLED
      EventPattern:
        source:
          - aws.elasticloadbalancing
        detail-type:
          - AWS API Call via CloudTrail
        detail:
          eventSource:
            - elasticloadbalancing.amazonaws.com
          eventName:
            - RegisterTargets
            - DeregisterTargets
          requestParameters:
            targetGroupArn:
              - !Ref SourceTargetGroupArn
      Targets:
        - Arn: !GetAtt AutoUpdateTargetGroupLambda.Arn
          Id: lambda

  AutoUpdateTargetLambdaInvokePermission:
    Type: AWS::Lambda::Permission
    Properties:
      Action: "lambda:InvokeFunction"
      FunctionName: !GetAtt AutoUpdateTargetGroupLambda.Arn
      Principal: events.amazonaws.com
      SourceArn: !GetAtt TargetGroupRegistrationEvents.Arn

  AutoUpdateTargetGroupLambdaRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              Service: lambda.amazonaws.com
            Action: "sts:AssumeRole"
      ManagedPolicyArns:
        - "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
      Policies:
        - PolicyName: AutoUpdateTargetGroupMembers
          PolicyDocument:
            Version: 2012-10-17
            Statement:
              - Effect: Allow
                Action:
                  - "elasticloadbalancing:RegisterTargets"
                  - "elasticloadbalancing:DeregisterTargets"
                Resource: !Ref DestinationTargetGroupArn
      Path: /service-role/

  AutoUpdateTargetGroupLambda:
    Type: AWS::Lambda::Function
    Properties:
      Description: Automatically (De)Registers destination target group(s) when a source target group changes
      Environment:
        Variables:
          DEST_ARN: !Ref DestinationTargetGroupArn
      Handler: index.lambda_handler
      MemorySize: 128
      Role: !GetAtt AutoUpdateTargetGroupLambdaRole.Arn
      Runtime: python3.6
      Timeout: 10
      Code:
        ZipFile: |
          import boto3
          import logging
          import os

          DEST_ARN = os.environ["DEST_ARN"]

          def lambda_handler(event, context):
            """
            Given the CloudWatch events 'RegisterTargets' or 'DeregisterTargets',
            replay (almost) the same events on another target group
            """

            # Initialize logger
            logger = logging.getLogger()
            logger.setLevel(logging.INFO)
            logger.debug(os.environ)
            logger.debug(event)
            logger.debug(context)

            # Instantiate API client for load balancers
            client = boto3.client("elbv2")

            # Gather relevant info from CloudWatch event
            event_name = event["detail"]["eventName"]
            event_targets = event["detail"]["requestParameters"]["targets"]

            # Log the relevant information from the received event
            logger.info("Received CloudWatch Event '%s' with the following targets:" % event_name)
            logger.info(event_targets)

            # Reformat / generalize targets
            targets = list(map(
              lambda target: {"Id": target["id"]},
              event_targets
            ))

            # Register new targets
            if event_name == "RegisterTargets":
              response = client.register_targets(
                TargetGroupArn=DEST_ARN,
                Targets=targets
              )

            # Or Deregister old targets
            elif event_name == "DeregisterTargets":
              response = client.deregister_targets(
                TargetGroupArn=DEST_ARN,
                Targets=targets
              )

            # Raise an exception if the Lambda received an unanticipated event
            else:
              raise ValueError("Unrecognized event type")

            return {
              "statusCode": 200,
              "body": {
                "event_processed": event_name,
                "response": response
              }
            }

This was only tested with awsvpc ECS tasks and ip (not instance) target groups. It works by capturing the register / deregister events from one target group then registering the same target on another target group which has a different port already configured. The cloudformation template will not bring the target groups up-to-date on instantiation, so either manually register the initial targets or re-register the targets by temporarily killing the appropriate service tasks.

For any who find this gist in the future, as of July 30 2019 AWS ECS now has this feature built-in: https://aws.amazon.com/about-aws/whats-new/2019/07/amazon-ecs-services-now-support-multiple-load-balancer-target-groups/ .

This solution is still useful in 2022 if your ECS service needs more than 5 allowed target groups per service for NLB.

One advice – turn on (enable) AWS CloudTrail before using this solution, overwise de/register events won't be invoked and your Lambda will never be called. It is sufficient to enable it in only one region with a multi-region setting ON.

Thanks for the awesome work @arpaulnet !