blog.md

Getting started with kubernetes cluster on AWS

AWS provides comprehensive guide to start with EKS on a link https://docs.aws.amazon.com/eks/latest/userguide/getting-started.html

Next steps generally follow that guide, difference is that provisioning play is (can be) implemented purely with terraform w/o cloud templates.

It has advantage in a way how we do deployments, but in your scenario cloud templates might have advantage too.

First step is creating a VPC with Public and Private Subnets for Your Amazon EKS Cluster

The simpliest VPC would be all subnets public, like seen on cloud template ( https://amazon-eks.s3-us-west-2.amazonaws.com/1.10.3/2018-06-05/amazon-eks-vpc-sample.yaml )

But if your goal is to reuse that in a production, the reasonable would be VPC with private and public parts, like https://docs.aws.amazon.com/eks/latest/userguide/create-public-private-vpc.html

At the moment there are limitations: https://docs.aws.amazon.com/eks/latest/userguide/network_reqs.html

Let's proceed with recommended network architecture that uses private subnets for your worker nodes and public subnets for Kubernetes to create internet-facing load balancers within

Step 1.1: Create an Elastic IP Address for Your NAT Gateway(s)

Worker nodes in private subnets require a NAT gateway for outbound internet access. A NAT gateway requires an Elastic IP address in your public subnet, but the VPC wizard does not create one for you. Create the Elastic IP address before running the VPC wizard.

To create an Elastic IP address

Open the Amazon VPC console at https://console.aws.amazon.com/vpc/.

In the left navigation pane, choose Elastic IPs. Choose Allocate new address, Allocate, Close. Note the Allocation ID for your newly created Elastic IP address; you enter this later in the VPC wizard.

https://gist.github.com/90f578d7080e5911fe0e16c4f1c3ce8b

Step 1.2: Run the VPC Wizard

The VPC wizard automatically creates and configures most of your VPC resources for you.

To run the VPC wizard

In the left navigation pane, choose VPC Dashboard. Choose Start VPC Wizard, VPC with Public and Private Subnets, Select. For VPC name, give your VPC a unique name. For Elastic IP Allocation ID, choose the ID of the Elastic IP address that you created earlier. Choose Create VPC. When the wizard is finished, choose OK. Note the Availability Zone in which your VPC subnets were created. Your additional subnets should be created in a different Availability Zone. Appropriate terraform part would be

https://gist.github.com/8757d05096262d1a7f8868b901770810

Step 2 create your Amazon EKS service role

Open the IAM console at https://console.aws.amazon.com/iam/.

Choose Roles, then Create role.

Choose EKS from the list of services, then Allows Amazon EKS to manage your clusters on your behalf for your use case, then Next: Permissions.

Choose Next: Review.

For Role name, enter a unique name for your role, such as eksServiceRole, then choose Create role.

Terraform definition for the role goes as below.

https://gist.github.com/70e5cf526f2c34dd3dd8376bc421d8ce

Step 3: Create a Control Plane Security Group

When you create an Amazon EKS cluster, your cluster control plane creates elastic network interfaces in your subnets to enable communication with the worker nodes. You should create a security group that is dedicated to your Amazon EKS cluster control plane, so that you can apply inbound and outbound rules to govern what traffic is allowed across that connection. When you create the cluster, you specify this security group, and that is applied to the elastic network interfaces that are created in your subnets.

To create a control plane security group

In the left navigation pane, for Filter by VPC, select your VPC and choose Security Groups, Create Security Group.

Note

If you don't see your new VPC here, refresh the page to pick it up. Fill in the following fields and choose Yes, Create:

For Name tag, provide a name for your security group. For example, -control-plane. For Description, provide a description of your security group to help you identify it later. For VPC, choose the VPC that you are using for your Amazon EKS cluster.

https://gist.github.com/ff26330f78e0ed2cf3d7d6d81923485e

Download aws authenticator, kind of

https://gist.github.com/510093d5b2fd229b51ccb60217ffa74b

Validate by running

https://gist.github.com/eb4c85fcc64e2eb7b6e612c11d766a6d

Check if you have latest awscli installed, you should get output similar to one below https://gist.github.com/ecbcf4ac3abaaa45088330c3151a32ce

Step 4: create EKS cluster

Now it is time to launch the cluster, either via UI or using provisioning tool of your choice

Making cluster to ECS might seem a bit tricky , as by default AWS proposes to hold cluster info in the separate files, like ~/.kube/eksconfig and use environment variables for switching context export KUBECONFIG=~/.kube/eksconfig

If you have multiple clusters configured, that's for sure will not be convenient to you.

https://gist.github.com/beb985e23c70c191aba8fcb9597bde79

for example above, you could validate, if you configured your cluster and kubectl right.

https://gist.github.com/5a1433476be343045a63fcf6346ca69a

Terraform part for the creating cluster action would be

https://gist.github.com/9b5e8af6857a124340fdfd2832d7db3d

Note, that instead of patching kubelet config, as per original guide, you can just get ready-to-use file content from terraform output kubeconfig.

Step 5,6: Now it is time to launch worker nodes

At a time being AWS recommends to launch stack using, for example, following cloud formation template https://amazon-eks.s3-us-west-2.amazonaws.com/1.10.3/2018-06-05/amazon-eks-nodegroup.yaml

What you would need from outcomes, is record the NodeInstanceRole for the node group that was created. You need this when you configure your Amazon EKS worker nodes.

On outputs pane you would see output kind of

https://gist.github.com/aca46bff4ae5d850ef699e2e737f8f97

final step would be ensuring that worker nodes can join your cluster.

You would need template for kubernetes configuration map

https://amazon-eks.s3-us-west-2.amazonaws.com/1.10.3/2018-06-05/aws-auth-cm.yaml

https://gist.github.com/3e3b8046462e4ef62a006ca59c3e0341

amend role arn and apply changes with

https://gist.github.com/e9d51f28888f736d774b5c89bd7802f0

In a few minutes your fleet should be ready

https://gist.github.com/69765902379bf1189e83cc0a5ed9725f

If you want to omit relying on cloud formation template by URL, and instead stick to fully scripted infrastructure:

Step 5: on that we prepare worker node and security groups

https://gist.github.com/acb86c7a743ed3c7339c87b508697f53

Step 5: on that step we are creating autoscaling group

https://gist.github.com/0641c95fb7f1639e825f77000d44c69b

On that step, you should also have fully working EKS cluster. Upon terraform apply run you should see smth like

https://gist.github.com/53966fb76cdd1e9b8bb67a7e614d3080

To make cluster fully operational, after provision you should:

1. get cluster kubelet config via

https://gist.github.com/6e4de48c9ed475b9462df285a9999fd5

after that you should be able to execute commands over your eks cluster

https://gist.github.com/3d7be36ee19a83fa2e6f5fb11ea4bef2

2. to allow nodes to join, you need to provide cluster with additional config map

https://gist.github.com/351b27929ec912dc765e26ac6de565d1

3. You should be able to run pods and services

https://gist.github.com/53ecb7eebd61c1927f192a077378b76a

Questions?

Troubleshouting ...

Now lets start smth

https://gist.github.com/2f83b4ede9a93deccca103ea61d82ca5

if you see state pending

https://gist.github.com/6d8ce44b134bc44b2822e353c88d4054

you might want immediately check

https://gist.github.com/2ba01ecfbd8e86b628a59fda8d03aebf

you might see smth like

https://gist.github.com/d7573ac3ad38563b074e823ac3e780f3

where in message you will get hint about the issue

Troubleshouting hints from AWS can be found on

https://docs.aws.amazon.com/eks/latest/userguide/troubleshooting.html