Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
AWS ECS and ECR deployment via Docker and Gitlab CI
image: docker:latest
variables:
REPOSITORY_URL: <AWS ACCOUNT ID>.dkr.ecr.eu-central-1.amazonaws.com/<ECS REPOSITORY NAME>
REGION: eu-central-1
TASK_DEFINTION_NAME: <TASK DEFINITION NAME>
CLUSTER_NAME: <CLUSTER NAME>
SERVICE_NAME: <SERVICE NAME>
services:
- docker:dind
before_script:
- apk add --no-cache curl jq python py-pip
- pip install awscli
- $(aws ecr get-login --no-include-email --region "${REGION}")
- IMAGE_TAG="$(echo $CI_COMMIT_SHA | head -c 8)"
stages:
- build
- deploy
build:
stage: build
script:
- echo "Building image..."
- docker build -t $REPOSITORY_URL:latest .
- echo "Tagging image..."
- docker tag $REPOSITORY_URL:latest $REPOSITORY_URL:$IMAGE_TAG
- echo "Pushing image..."
- docker push $REPOSITORY_URL:latest
- docker push $REPOSITORY_URL:$IMAGE_TAG
only:
- master
deploy:
stage: deploy
script:
- echo $REPOSITORY_URL:$IMAGE_TAG
- TASK_DEFINITION=$(aws ecs describe-task-definition --task-definition "$TASK_DEFINTION_NAME" --region "${REGION}")
- NEW_CONTAINER_DEFINTIION=$(echo $TASK_DEFINITION | python $CI_PROJECT_DIR/update_task_definition_image.py $REPOSITORY_URL:$IMAGE_TAG)
- echo "Registering new container definition..."
- aws ecs register-task-definition --region "${REGION}" --family "${TASK_DEFINTION_NAME}" --container-definitions "${NEW_CONTAINER_DEFINTIION}"
- echo "Updating the service..."
- aws ecs update-service --region "${REGION}" --cluster "${CLUSTER_NAME}" --service "${SERVICE_NAME}" --task-definition "${TASK_DEFINTION_NAME}"
only:
- master
import sys, json, argparse
parser = argparse.ArgumentParser('Replaces image in the task definition')
parser.add_argument('image_uri', metavar='I', type=str, nargs='+',
help='The new image URI')
args = parser.parse_args()
definition = json.load(sys.stdin)['taskDefinition']['containerDefinitions']
definition[0]['image'] = args.image_uri[0]
print json.dumps(definition)
@jobrienski

This comment has been minimized.

Copy link

@jobrienski jobrienski commented Sep 11, 2018

- TASK_DEFINITION=$(aws ecs describe-task-definition --task-definition "$TASK_DEFINTION_NAME" --region "${REGION}")

should be $TASK_DEFINITION_NAME

@hopenbr

This comment has been minimized.

Copy link

@hopenbr hopenbr commented Apr 18, 2019

to avoid python you can use jq

- NEW_CONTAINER_DEFINTIION=$(echo $TASK_DEFINITION | jq --arg IMAGE "$REPOSITORY_URL:$IMAGE_TAG" '.taskDefinition.containerDefinitions[0].image = $IMAGE | .taskDefinition.containerDefinitions[0]')

@philipgiuliani

This comment has been minimized.

Copy link

@philipgiuliani philipgiuliani commented Jul 19, 2019

If you have volumes you can add the following:

- VOLUMES=$(echo $TASK_DEFINITION | jq '.taskDefinition.volumes')
- aws ecs register-task-definition --region "${REGION}" --family "${TASK_DEFINTION_NAME}" --container-definitions "${NEW_CONTAINER_DEFINTIION} --volumes "${VOLUMES}"
@tsuz

This comment has been minimized.

Copy link

@tsuz tsuz commented Aug 14, 2019

- IMAGE_TAG="$(echo $CI_COMMIT_SHA | head -c 8)"

could also be replaced by

- IMAGE_TAG="$CI_COMMIT_SHORT_SHA"

Reference

@davehodg

This comment has been minimized.

Copy link

@davehodg davehodg commented Sep 29, 2019

I'm getting:

Pushing image...
$ docker push $REPOSITORY_URL:latest
The push refers to repository [733970770342.dkr.ecr.eu-west-2.amazonaws.com/nettools]
729283870dfa: Preparing
6ead9a587789: Preparing
1e0f48f41d21: Preparing
3bfeb766f97b: Preparing
ea1227feeccb: Preparing
9cae1895156d: Preparing
52dba9daa22c: Preparing
78c1b9419976: Preparing
no basic auth credentials

The internet's not being helpful. I've made a few suggested mods.

@kamihouse

This comment has been minimized.

Copy link

@kamihouse kamihouse commented Oct 7, 2019

@davehodg try custom this:

Login: $(aws ecr get-login --no-include-email --region "${AWS_DEFAULT_REGION}")

variables:
  DOCKER_DRIVER: "overlay2"
  DOCKER_TLS_CERTDIR: ""
  DOCKER_AUTH_CONFIG: "{\"credsStore\": \"ecr-login\"}"
@davehodg

This comment has been minimized.

Copy link

@davehodg davehodg commented Oct 8, 2019

- docker build -t xxx .
- docker tag xxx:latest 733970770999.dkr.ecr.eu-west-2.amazonaws.com/xxx:latest
- echo "Pushing image..."
- docker push 733970770999.dkr.ecr.eu-west-2.amazonaws.com/nettools:latest
- $(aws ecr get-login --no-include-email --region eu-west-2| sed 's|https://||')
- aws ecs update-service --service xx-container-service --task-definition task-definition:20 --cluster xxx-cluster

Works fine!

@hikmahgumelar

This comment has been minimized.

Copy link

@hikmahgumelar hikmahgumelar commented Nov 19, 2019

Nice......

@kutzhanov

This comment has been minimized.

Copy link

@kutzhanov kutzhanov commented Dec 20, 2019

You can also add the following commands to check if the previous task was successfully stopped and the new one is started:

- TASK_ARN=$(aws ecs list-tasks --region "${REGION}" --cluster "${CLUSTER_NAME}" --service-name "${SERVICE_NAME}")
- TASK_ID=$(echo ${TASK_ARN:72:36})
- echo "Waiting until the previous task ${TASK_ID} is stopped..."
- aws ecs wait tasks-stopped --region "${REGION}" --cluster "${CLUSTER_NAME}" --tasks "${TASK_ID}"
- echo "The previous task ${TASK_ID} has stopped."
- TASK_ARN=$(aws ecs list-tasks --region "${REGION}" --cluster "${CLUSTER_NAME}" --service-name "${SERVICE_NAME}")
- TASK_ID=$(echo ${TASK_ARN:72:36})
- echo "New task ${TASK_ID} is running."
- aws ecs describe-tasks --region "${REGION}" --cluster "${CLUSTER_NAME}" --tasks "${TASK_ID}"
@ccetina

This comment has been minimized.

Copy link

@ccetina ccetina commented Jun 27, 2020

(y)

@guerzon

This comment has been minimized.

Copy link

@guerzon guerzon commented Aug 9, 2020

Hey, newbie question:

/bin/bash: line 102: apk: command not found

..isn't it support to be in the docker image?

@jlis

This comment has been minimized.

Copy link
Owner Author

@jlis jlis commented Aug 11, 2020

Hey, newbie question:

/bin/bash: line 102: apk: command not found

..isn't it support to be in the docker image?

@guerzon APK is a package manager used by Alpine Linux (for example). It depends what image you're using. For Ubuntu, you might use APT.

@guerzon

This comment has been minimized.

Copy link

@guerzon guerzon commented Aug 12, 2020

Hey, newbie question:
/bin/bash: line 102: apk: command not found
..isn't it support to be in the docker image?

@guerzon APK is a package manager used by Alpine Linux (for example). It depends what image you're using. For Ubuntu, you might use APT.

Yeah I'm aware of that, which was why I was kicking myself when I found out I was using another image in the step where this error showed up. Thanks, all good now.

@pavel-rezabek

This comment has been minimized.

Copy link

@pavel-rezabek pavel-rezabek commented Aug 19, 2020

you can also use --query parameter instead of jq to get specific output:
https://docs.aws.amazon.com/cli/latest/userguide/cli-usage-output.html#cli-usage-output-filter

@fgiudici95

This comment has been minimized.

Copy link

@fgiudici95 fgiudici95 commented Sep 4, 2020

Hey, newbie question:
/bin/bash: line 102: apk: command not found
..isn't it support to be in the docker image?

@guerzon APK is a package manager used by Alpine Linux (for example). It depends what image you're using. For Ubuntu, you might use APT.

Yeah I'm aware of that, which was why I was kicking myself when I found out I was using another image in the step where this error showed up. Thanks, all good now.

Hi @guerzon and @jlis
Can you share your gitlab-runner configuration?
I have the same problem as you.
Thanks

@guerzon

This comment has been minimized.

Copy link

@guerzon guerzon commented Sep 4, 2020

Hey, newbie question:
/bin/bash: line 102: apk: command not found
..isn't it support to be in the docker image?

@guerzon APK is a package manager used by Alpine Linux (for example). It depends what image you're using. For Ubuntu, you might use APT.

Yeah I'm aware of that, which was why I was kicking myself when I found out I was using another image in the step where this error showed up. Thanks, all good now.

Hi @guerzon and @jlis
Can you share your gitlab-runner configuration?
I have the same problem as you.
Thanks

Hi, here is my working configuration: https://gist.github.com/guerzon/3844df5399b28493136c940e0b5fbecd

Just to be clear, my issue was that the stage where I get this error was using another image, and it has nothing to do with my gitlab runner config. But hopefully the config works for you.

@tusharpatil985

This comment has been minimized.

Copy link

@tusharpatil985 tusharpatil985 commented Apr 8, 2021

Hello,

I am running the same in GitlabCI but getting the authentication error in below command:

aws ecs register-task-definition --region "${AWS_DEFAULT_REGION}" --family "${TASK_DEFINITION_NAME}" --container-definitions "${NEW_CONTAINER_DEFINTIION}"

Error :"An error occurred (ClientException) when calling the RegisterTaskDefinition operation: Private repository authentication requires that the task definition specify a value for executionRoleArn."

How to pass the credentials for private repo.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment