How to configure ECS EC2 instances to authenticate to JFrog Artifactory Docker Registry

README.md

How to configure ECS EC2 instances to authenticate to JFrog Artifactory Docker Registry

The JFrog documentation is lacking on how to do this in a proper way, expecting to use the normal procedures for logging into docker and not in an automated way. This gets even worse if you try and configure your task to authenticate to a private registry - which does not seem to be possible.

The solution is to adjust your user-data, preferably storing your config and key in Secrets Manager.

Putting the config into Secret Manager:

aws secretsmanager update-secret --secret-id artifactory --region us-west-1 --secret-string '{"https://companyname-repo-virtual.jfrog.io": {"auth": "AUTHKEY_FROM_SETMEUP","email": "EMAIL_FROM_SETMEUP"}}'

Adjusting your User Data script:

# Install awslogs and the jq JSON parser
yum install -y awslogs jq aws-cli

# ECS config
echo ECS_CLUSTER='${ECS_CLUSTER_NAME}' >> /etc/ecs/ecs.config
echo ECS_ENGINE_AUTH_TYPE=dockercfg
echo ECS_ENGINE_AUTH_DATA=`aws secretsmanager get-secret-value --secret-id artifactory --query SecretString --output text --region us-west-1` >> /etc/ecs/ecs.config

Which would evaluate to:

# cat /etc/ecs/ecs.config
ECS_CLUSTER=clustername
ECS_ENGINE_AUTH_TYPE=dockercfg
ECS_ENGINE_AUTH_DATA={"https://companyname-repo-virtual.jfrog.io": {"auth": "AUTHKEY_FROM_SETMEUP","email": "EMAIL_FROM_SETMEUP"}}

Sources

• https://docs.aws.amazon.com/AmazonECS/latest/developerguide/private-auth-container-instances.html#docker-auth-formats
• https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-agent-config.html

Very helpful thanks!

Here is another way, without the need for user_data: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/private-auth.html

Unfortunately this did not work for me.. I am using an ECS optimized AMI and passing the auth value both in a user-data format and also tested it by adding it to /etc/ecs/ecs.config manually. When running a docker pull it can't pull the image due to authorization but when I execute a docker login and put in the credentials its able to pull the image.

Hrm. Just checked what we are running: amzn-ami-*-amazon-ecs-optimized.
This is on continuous rotation, and the servers are replaced on a daily basis.

It sounds to me that something is wrong with the credentials stored in SecretsManager.
Hope this helps!

@cabrinha I tried getting it working with that - I can't remember what stopped me anymore as this was a year ago. But this was the only way I could get it working.

Did you manage to get it working that way?

@cabrinha I tried getting it working with that - I can't remember what stopped me anymore as this was a year ago. But this was the only way I could get it working.

Did you manage to get it working that way?

I actually haven't tried it yet myself. We're using Puppet to manage the /etc/ecs/ecs.config file. I just found the docs and brought it up as a suggestion. Sorry for the confusion 😅

@cabrinha Not at all! :) Thanks for getting back here.
I am close to 100% sure I went down that route and there was something spooky with it and Artifactory.
After many hours of trial, error, and headbanging this was the only solution that finally worked. :)

@hussfelt I actually removed secret manager out of the equation and directly put in the auth token into the user-data script for testing. It did not work. I will try a couple of other things and see if it works out..

@gdelgado strange. I can't think of any reason why that would not work - we have had multiple runs with Artifcatory credentials though, it's not at all clear which one to use at any given point and how they should be "encoded" or encoded at all. The credentials used here come straight from the "SETMEUP" guide on a repository in Artifactory. Hope this helps!

@hussfelt Just wanted to update this thread that the steps above do indeed work. The issue I was having was that after implementing the steps I was trying to execute a docker pull manually but as the env vars are configured via ECS it would try to use my local docker config instead of the ECS configured one which is done via the ECS Agent. Thanks for all the help on this..

@gdelgado thanks for following up for others reaching this thread!