A Kubernetes cronjob to refresh ECR authentication

create-aws-ecr-authentication-cronjob.md

A Kubernetes cronjob to refresh ECR authentication

Create AWS secret

kubectl create secret generic aws-secret --from-literal=AWS_ACCOUNT= --from-literal=AWS_ACCESS_KEY_ID= --from-literal=AWS_SECRET_ACCESS_KEY= --from-literal=AWS_DEFAULT_REGION= --from-literal=AWS_REGION=

Create cronjob

#aws-registry-credential-cron.yaml
apiVersion: batch/v1beta1
kind: CronJob
metadata:
  name: aws-registry-credential-cron
spec:
  schedule: "* */8 * * *"
  successfulJobsHistoryLimit: 2
  failedJobsHistoryLimit: 2  
  jobTemplate:
    spec:
      backoffLimit: 4
      template:
        spec:
          serviceAccountName: default
          terminationGracePeriodSeconds: 0
          restartPolicy: Never
          containers:
          - name: kubectl
            imagePullPolicy: IfNotPresent
            image: xynova/aws-kubectl:latest
            envFrom:
            - secretRef:
                name: aws-secret
            command:
            - "/bin/sh"
            - "-c"
            - |
              DOCKER_REGISTRY_SERVER=https://${AWS_ACCOUNT}.dkr.ecr.${AWS_REGION}.amazonaws.com
              DOCKER_USER=AWS
              DOCKER_PASSWORD=$(aws ecr get-login --region ${AWS_REGION} --registry-ids ${AWS_ACCOUNT} | cut -d' ' -f6)
              kubectl delete secret aws-registry || true
              kubectl create secret docker-registry aws-registry \
              --docker-server=$DOCKER_REGISTRY_SERVER \
              --docker-username=$DOCKER_USER \
              --docker-password=$DOCKER_PASSWORD \
              --docker-email=no@email.local
              kubectl patch serviceaccount default -p '{"imagePullSecrets":[{"name":"aws-registry"}]}'

Run

kubectl create -f aws-registry-credential-cron.yaml

# trigger the first run
kubectl create job --from=cronjob/aws-registry-credential-cron aws-registry-credential-cron-manual-001
kubectl logs job/aws-registry-credential-cron-manual-001
secret "aws-registry" deleted
secret "aws-registry" created
serviceaccount "default" not patched

Error

While running the kubectl jobs, I got the below error.

Error from server (Forbidden): secrets "aws-registry" is forbidden: User "system:serviceaccount:sam-tst-ns:default" cannot delete resource "secrets" in API group "" in the namespace "NAMESPACE"
Error from server (Forbidden): secrets is forbidden: User "system:serviceaccount:sam-tst-ns:default" cannot create resource "secrets" in API group "" in the namespace "NAMESPACE"
Error from server (Forbidden): serviceaccounts "default" is forbidden: User "system:serviceaccount:sam-tst-ns:default" cannot get resource "serviceaccounts" in API group "" in the namespace "NAMESPACE"

Fix

Got a fix from here Azure/AKS#954

create role.yaml

kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  namespace: NAMESPACE
  name: repo-creator
rules:
- apiGroups: ["", "extensions", "apps"]
  resources: ["deployments", "replicasets", "pods", "services", "serviceaccounts", "secrets"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: repo-creator
  namespace: NAMESPACE
subjects:
- kind: Group
  name: system:serviceaccounts
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: repo-creator
  apiGroup: rbac.authorization.k8s.io

save the file

run below

kubectl apply -f role.yaml

then, run the kubectl commands to create the cronjob, and check the logs. It works fine now.