Deploy Express API - Kubernetes (EKS) with HTTPS endpoint

README.md

Important: This tutorial assume you already has installed and know how to use, aws cli, kubectl and eksctl

Create and Setup Cluster and required policies

1. Create Fargate Cluster

eksctl create cluster --region us-west-1 --name express-api --version 1.25 --fargate

Note: This command create a stack in cloudformation

2. Enable cluster to use IAM

eksctl utils associate-iam-oidc-provider --region us-west-1 --cluster express-api --approve

3. Download the IAM policy to allow AWS Load Balancer Controller to make requests to AWS API's

curl -o iam_policy.json https://raw.githubusercontent.com/kubernetes-sigs/aws-load-balancer-controller/v2.4.4/docs/install/iam_policy.json

4. Create an IAM policy with the file downloaded in step 3

aws iam create-policy \
   --policy-name AWSLoadBalancerControllerIAMPolicy \
   --policy-document file://iam_policy.json

5. To create a service account named aws-load-balancer-controller in the kube-system namespace for the AWS Load Balancer Controller, run the following command:

eksctl create iamserviceaccount \
  --cluster=express-api \
  --namespace=kube-system \
  --name=aws-load-balancer-controller \
  --attach-policy-arn=arn:aws:iam::{org-id}:policy/AWSLoadBalancerControllerIAMPolicy \
  --override-existing-serviceaccounts \
  --approve \
  --region us-west-1

Note: This command create a stack in cloudformation

6. To verify that the new service role is created, run one of the following commands:

eksctl get iamserviceaccount --region us-west-1 --cluster express-api --name aws-load-balancer-controller --namespace kube-system
#or
kubectl get serviceaccount aws-load-balancer-controller --namespace kube-system

Install the AWS Load Balancer Controller using Helm

1. Add the Amazon EKS chart repo to Helm

helm repo add eks https://aws.github.io/eks-charts

2. Install the TargetGroupBinding custom resource definitions (CRDs)

kubectl apply -k "github.com/aws/eks-charts/stable/aws-load-balancer-controller//crds?ref=master"

3. Install helm chart

helm install aws-load-balancer-controller eks/aws-load-balancer-controller \
    --set clusterName=express-api \
    --set serviceAccount.create=false \
    --set region=us-west-1 \
    --set vpcId={vpc-id} \
    --set serviceAccount.name=aws-load-balancer-controller \
    -n kube-system

Setup AWS Load Balancer Controller

1. Create a Fargate profile

eksctl create fargateprofile --cluster express-api --region us-west-1 --name express-api-profile --namespace express-api-namespace

2. Deploy YML file

kubectl apply -f express-api-alb.yml

Note: If you want https on your load balancer, you need to create it before in the same region when you are deployed your cluster (us-west-1 for this example), then pass the certificate arn in the file.yml

3. Verify that the Ingress resource was created

kubectl get ingress/express-api-ingress -n express-api-namespace
#or
kubectl get ingresses.networking.k8s.io express-api-ingress -n express-api-namespace

Output:

NAME                  CLASS   HOSTS   ADDRESS                                                                 PORTS   AGE
express-api-ingress   alb     *       k8s-expressa-expressa-xxxxxxxxxx-xxxxxxxx.us-west-2.elb.amazonaws.com   80      12m

Note: If your Ingress isn't created after several minutes, view the AWS Load Balancer Controller logs by running the following command:

kubectl logs -n kube-system deployment.apps/aws-load-balancer-controller

4. Open the browser and paste the load balancer url or your custom domain

Note: Remember create an A record in route 53 or your domain admin point to load balancer

5. Scale your deployment

kubectl scale deployments express-api-deployment --replicas=3 -n express-api-namespace

References

https://repost.aws/knowledge-center/eks-alb-ingress-controller-fargate

https://aws.amazon.com/blogs/containers/how-to-expose-multiple-applications-on-amazon-eks-using-a-single-application-load-balancer/

Delete Cluster

eksctl delete cluster --region us-west-1 --name express-api 

express-api-alb.yml

---
apiVersion: v1
kind: Namespace
metadata:
  name: express-api-namespace
---
apiVersion: apps/v1
kind: Deployment
metadata:
  namespace: express-api-namespace
  name: express-api-deployment
spec:
  selector:
    matchLabels:
      app.kubernetes.io/name: express-api-deployment
  replicas: 2
  template:
    metadata:
      labels:
        app.kubernetes.io/name: express-api-deployment
    spec:
      containers:
        - name: express-api-container
          image: diegofcornejo/express-basic-api:latest
          ports:
            - containerPort: 8080
          resources:
            requests:
              memory: "256Mi"
              cpu: "250m"
            limits:
              memory: "256Mi"
              cpu: "250m"
---
apiVersion: apps/v1
kind: Deployment
metadata:
  namespace: express-api-namespace
  name: express-api-deployment-high
spec:
  selector:
    matchLabels:
      app.kubernetes.io/name: express-api-deployment-high
  replicas: 0
  template:
    metadata:
      labels:
        app.kubernetes.io/name: express-api-deployment-high
    spec:
      containers:
        - name: express-api-container-high
          image: diegofcornejo/express-basic-api:latest
          ports:
            - containerPort: 8080
          resources:
            requests:
              memory: "512Mi"
              cpu: "1000m"
            limits:
              memory: "512Mi"
              cpu: "1000m"
---
apiVersion: v1
kind: Service
metadata:
  namespace: express-api-namespace
  name: express-api-service
spec:
  type: NodePort
  selector:
    app.kubernetes.io/name: express-api-deployment
  ports:
    - port: 80
      targetPort: 8080
      protocol: TCP
---
apiVersion: v1
kind: Service
metadata:
  namespace: express-api-namespace
  name: express-api-service-high
spec:
  type: NodePort
  selector:
    app.kubernetes.io/name: express-api-deployment-high
  ports:
    - port: 80
      targetPort: 8080
      protocol: TCP
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  namespace: express-api-namespace
  name: express-api-ingress
  annotations:
    # kubernetes.io/ingress.class: alb (default)
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip
    # alb.ingress.kubernetes.io/healthcheck-protocol: HTTP
    # alb.ingress.kubernetes.io/healthcheck-port: traffic-port
    # alb.ingress.kubernetes.io/healthcheck-interval-seconds: '15'
    # alb.ingress.kubernetes.io/healthcheck-timeout-seconds: '5'
    # alb.ingress.kubernetes.io/success-codes: '200'
    # alb.ingress.kubernetes.io/healthy-threshold-count: '2'
    # alb.ingress.kubernetes.io/unhealthy-threshold-count: '2'
    alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:us-west-1:{org-id}:certificate/{certificate-uuid}
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS":443}]'
      # Set HTTP to HTTPS redirects. Every HTTP listener configured will be redirected to below mentioned port over HTTPS.
    alb.ingress.kubernetes.io/tls-redirect: "443"
spec:
  ingressClassName: alb
  rules:
    #  - host: fargate.eks.api.ubisuite.com #Set host(s) if you use more than one host
    #    http:
    #     paths:
    #       - path: /
    #         pathType: Prefix
    #         backend:
    #           service:
    #             name: express-api-service
    #             port:
    #               number: 80
    #       - path: /
    #         pathType: Prefix
    #         backend:
    #           service:
    #             name: express-api-service-high
    #             port:
    #               number: 80
    #       - path: /high
    #         pathType: Prefix
    #         backend:
    #           service:
    #             name: express-api-service-high
    #             port:
    #               number: 80
    - http:
        paths:
          - path: / #All request can be processed by all target groups (services)
            pathType: Prefix
            backend:
              service:
                name: express-api-service
                port:
                  number: 80
          - path: /
            pathType: Prefix
            backend:
              service:
                name: express-api-service-high
                port:
                  number: 80
          - path: /high #This specific path can be processed just for high service
            pathType: Prefix
            backend:
              service:
                name: express-api-service-high
                port:
                  number: 80

iam_policy.json

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "iam:CreateServiceLinkedRole"
            ],
            "Resource": "*",
            "Condition": {
                "StringEquals": {
                    "iam:AWSServiceName": "elasticloadbalancing.amazonaws.com"
                }
            }
        },
        {
            "Effect": "Allow",
            "Action": [
                "ec2:DescribeAccountAttributes",
                "ec2:DescribeAddresses",
                "ec2:DescribeAvailabilityZones",
                "ec2:DescribeInternetGateways",
                "ec2:DescribeVpcs",
                "ec2:DescribeVpcPeeringConnections",
                "ec2:DescribeSubnets",
                "ec2:DescribeSecurityGroups",
                "ec2:DescribeInstances",
                "ec2:DescribeNetworkInterfaces",
                "ec2:DescribeTags",
                "ec2:GetCoipPoolUsage",
                "ec2:DescribeCoipPools",
                "elasticloadbalancing:DescribeLoadBalancers",
                "elasticloadbalancing:DescribeLoadBalancerAttributes",
                "elasticloadbalancing:DescribeListeners",
                "elasticloadbalancing:DescribeListenerCertificates",
                "elasticloadbalancing:DescribeSSLPolicies",
                "elasticloadbalancing:DescribeRules",
                "elasticloadbalancing:DescribeTargetGroups",
                "elasticloadbalancing:DescribeTargetGroupAttributes",
                "elasticloadbalancing:DescribeTargetHealth",
                "elasticloadbalancing:DescribeTags"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "cognito-idp:DescribeUserPoolClient",
                "acm:ListCertificates",
                "acm:DescribeCertificate",
                "iam:ListServerCertificates",
                "iam:GetServerCertificate",
                "waf-regional:GetWebACL",
                "waf-regional:GetWebACLForResource",
                "waf-regional:AssociateWebACL",
                "waf-regional:DisassociateWebACL",
                "wafv2:GetWebACL",
                "wafv2:GetWebACLForResource",
                "wafv2:AssociateWebACL",
                "wafv2:DisassociateWebACL",
                "shield:GetSubscriptionState",
                "shield:DescribeProtection",
                "shield:CreateProtection",
                "shield:DeleteProtection"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "ec2:AuthorizeSecurityGroupIngress",
                "ec2:RevokeSecurityGroupIngress"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "ec2:CreateSecurityGroup"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "ec2:CreateTags"
            ],
            "Resource": "arn:aws:ec2:*:*:security-group/*",
            "Condition": {
                "StringEquals": {
                    "ec2:CreateAction": "CreateSecurityGroup"
                },
                "Null": {
                    "aws:RequestTag/elbv2.k8s.aws/cluster": "false"
                }
            }
        },
        {
            "Effect": "Allow",
            "Action": [
                "ec2:CreateTags",
                "ec2:DeleteTags"
            ],
            "Resource": "arn:aws:ec2:*:*:security-group/*",
            "Condition": {
                "Null": {
                    "aws:RequestTag/elbv2.k8s.aws/cluster": "true",
                    "aws:ResourceTag/elbv2.k8s.aws/cluster": "false"
                }
            }
        },
        {
            "Effect": "Allow",
            "Action": [
                "ec2:AuthorizeSecurityGroupIngress",
                "ec2:RevokeSecurityGroupIngress",
                "ec2:DeleteSecurityGroup"
            ],
            "Resource": "*",
            "Condition": {
                "Null": {
                    "aws:ResourceTag/elbv2.k8s.aws/cluster": "false"
                }
            }
        },
        {
            "Effect": "Allow",
            "Action": [
                "elasticloadbalancing:CreateLoadBalancer",
                "elasticloadbalancing:CreateTargetGroup"
            ],
            "Resource": "*",
            "Condition": {
                "Null": {
                    "aws:RequestTag/elbv2.k8s.aws/cluster": "false"
                }
            }
        },
        {
            "Effect": "Allow",
            "Action": [
                "elasticloadbalancing:CreateListener",
                "elasticloadbalancing:DeleteListener",
                "elasticloadbalancing:CreateRule",
                "elasticloadbalancing:DeleteRule"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "elasticloadbalancing:AddTags",
                "elasticloadbalancing:RemoveTags"
            ],
            "Resource": [
                "arn:aws:elasticloadbalancing:*:*:targetgroup/*/*",
                "arn:aws:elasticloadbalancing:*:*:loadbalancer/net/*/*",
                "arn:aws:elasticloadbalancing:*:*:loadbalancer/app/*/*"
            ],
            "Condition": {
                "Null": {
                    "aws:RequestTag/elbv2.k8s.aws/cluster": "true",
                    "aws:ResourceTag/elbv2.k8s.aws/cluster": "false"
                }
            }
        },
        {
            "Effect": "Allow",
            "Action": [
                "elasticloadbalancing:AddTags",
                "elasticloadbalancing:RemoveTags"
            ],
            "Resource": [
                "arn:aws:elasticloadbalancing:*:*:listener/net/*/*/*",
                "arn:aws:elasticloadbalancing:*:*:listener/app/*/*/*",
                "arn:aws:elasticloadbalancing:*:*:listener-rule/net/*/*/*",
                "arn:aws:elasticloadbalancing:*:*:listener-rule/app/*/*/*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "elasticloadbalancing:ModifyLoadBalancerAttributes",
                "elasticloadbalancing:SetIpAddressType",
                "elasticloadbalancing:SetSecurityGroups",
                "elasticloadbalancing:SetSubnets",
                "elasticloadbalancing:DeleteLoadBalancer",
                "elasticloadbalancing:ModifyTargetGroup",
                "elasticloadbalancing:ModifyTargetGroupAttributes",
                "elasticloadbalancing:DeleteTargetGroup"
            ],
            "Resource": "*",
            "Condition": {
                "Null": {
                    "aws:ResourceTag/elbv2.k8s.aws/cluster": "false"
                }
            }
        },
        {
            "Effect": "Allow",
            "Action": [
                "elasticloadbalancing:RegisterTargets",
                "elasticloadbalancing:DeregisterTargets"
            ],
            "Resource": "arn:aws:elasticloadbalancing:*:*:targetgroup/*/*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "elasticloadbalancing:SetWebAcl",
                "elasticloadbalancing:ModifyListener",
                "elasticloadbalancing:AddListenerCertificates",
                "elasticloadbalancing:RemoveListenerCertificates",
                "elasticloadbalancing:ModifyRule"
            ],
            "Resource": "*"
        }
    ]
}