This document provides a rapid-fire overview of Kubernetes concepts, vocabulary, and operations. The target audience is anyone who runs applications in a cloud environment today, and who wants to understand the basic mechanics of a Kubernetes cluster. The goal is that within 10 minutes, managers who read this should be able to listen in on a Kubernetes conversation and follow along at a high level, and engineers should be ready to deploy a sample app to a toy cluster of their own.
This orientation doc was written because the official Kubernetes docs are a great reference, but they present a small cliff to climb for newcomers.
If you want to understand why you should consider running Kubernetes, see the official Kubernetes conceptual overview document. This document is intended to complement that one, but one layer deeper.
For a deep dive, see Kubernetes concepts.
Kubernetes can run on almost any server, whether physical, cloud, or in a virtual machine on your laptop. Each server in your cluster is called a Node. Nodes are either a Master Node or a Worker Node. Your application runs on Worker Nodes, while the Master Nodes run services to manage the cluster.
Kubernetes is declarative, meaning that when you want a piece of infrastructure to exist (like a certain app running in your cluster), you first create a description of that object and store it inside a database provided by the Master Nodes. Kubernetes constantly compares the objects in the database to the real world, and will launch, shutdown, or otherwise manage your applications until the real world matches the description in the database. This means that if anything changes in the real world, like if a Worker Node dies or your app crashes, then Kubernetes will react to restore your system to the desired state.
The Kubernetes database is exposed via an API. Applications running on your cluster can talk to the Kubernetes API to query or modify the infrastructure that they're running on, which enables CI/CD, Autoscaling, and other dynamic infrastructure.
The parts of Kubernetes that watch the database for changes and then execute those changes are called Controllers. For example, Kubernetes can manage AWS ELBs through a Controller. Most Controllers are hosted inside Kubernetes as just another application. This effectively provides a plugin system which allows you to adapt Kubernetes to any environment by writing your own Controllers to interact with the Kubernetes API.
To run applications on Kubernetes, you must package them as Docker containers. The Worker Nodes are responsible for running your Docker containers inside an abstraction called a Pod. You can think of a Pod as very similar to a traditional server or VM:
- Applications inside a Pod can talk to each other on
localhost, and must not overlap ports. - Applications inside a Pod can communicate over other traditional methods, like shared memory.
- Each Pod gets its own IP address.
- Pods talk to each other on their IP addresses.
- You can run multiple copies of each Pod for scaling and high-availability, and you should run separate Pods for each application component rather than piling all of your applications into one.
Kubernetes offers elaborate methods for mapping your Pods onto Nodes, to ensure that your application has the hardware resources and network layout that it needs. These mappings are named for different types of "Sets", eg "ReplicaSet", "DaemonSet", etc. These Sets provide different strategies in how Kubernetes will manage your applications. For example, if you want to ensure that one (and only one) copy of an application is running on each Node, then you want a DaemonSet.
All Pods exist inside a Namespace. This is similar to namespaces in code - it's meant to provide isolation and clarity. For this purpose, think of Namespaces as similar to a VPC or a private network. Example Namespaces could be different environments (dev / qa / prod), or even different teams or products, all running on the same Kubernetes cluster.
Most objects in the Kubernetes world, like Pods and Nodes, can be assigned tags of your own design. Tags are called Labels in Kubernetes. Labels can be used to direct many Kubernetes operations, like which Pods run on which Nodes. For example, you could add some GPU servers to your pool of Worker Nodes, and then apply a Label to those Nodes of "gpu=true". An application which requires a GPU could then request to be run on a Node where "gpu=true". (This is known as affinity.) Label-based configuration is used extensively.
Kubernetes also provides simple Configuration Management. Config files and simple values (integers, booleans, etc) can be uploaded into an object called a ConfigMap. The contents of a ConfigMap can then be mapped into the filesystem of your application, or into environment variables for the Pod where your application runs.
Very similar to a ConfigMap is another configuration object called a Secret. Secrets provide the same features as ConfigMaps, but are encrypted on disk inside the Master Nodes.
To deploy all the pieces of your application, you create a Deployment object. Inside a Deployment, you specify all the objects needed to deploy your app: a Pod definition (known as a "PodSpec"); the list of containers that should be running in the Pod; any ConfigMaps or Secrets necessary; how many copies ("Replicas") of the Pod should be running in your cluster; and how those Replicas should be distributed across your cluster (eg, with a "ReplicaSet" or perhaps a "DaemonSet"). The Deployment object will handle a rolling deploy for you, including an automatic rollback if necessary.
Kubernetes leverages many virtual-networking features of Linux, effectively creating an isolated private network inside the cluster. For example, Pods are assigned an IP address (called the Cluster IP) which is not routable anywhere except within the cluster. Cluster operators rarely need to know anything about the internal network layout, as Kubernetes handles all of the details, including how traffic gets into or out of the cluster network. (More below.)
Pods are ephemeral in Kubernetes. Meaning, if you deploy a new version of an application, or even just restart it, it is likely that the current Pod where it is running will be destroyed and a new one created in its place with a different Cluster IP. Therefore, even though Pods could find each other by querying the Pods API and speaking directly to a Pod's Cluster IP, it's not recommended since that list will change very often.
Instead of configuring your application to communicate over the Pod Cluster IPs directly, you declare a Service. A Service is like a load balancer - it gets allocated a static Cluster IP, and it will distribute incoming traffic across a set of Pods. A Service will watch the Pods API for you, and select a constantly-updated list of target Pods according to a "selector" that you provide, which filters Pods by Labels.
Each Pod that a Service identifies as a target is called an Endpoint. Services can also be defined with manual Endpoints instead of a Pod selector, so that the Service points to some external application. This allows your application to communicate exclusively with Services, and provides some degree of service discovery.
Services are automatically exposed to your application via internal DNS, so that your application can simply connect to http://mybackend or similar. Your application containers will also have an environment variable set for each Service, eg MYBACKEND_SERVICE_HOST and MYBACKEND_SERVICE_PORT, but this is only true for containers launched after the Service was defined, so DNS is recommended.
Services are normally accessible only from inside the cluster. However, a Service can also be exposed externally. On AWS, Kubernetes will create and configure an ELB for you. Traffic will get routed to the correct Pod, even if it's not running on whatever Node happened to receive the request from the ELB.
New versions of Kubernetes also introduce a networking object called an Ingress. An Ingress today is primarily responsible for managing your Layer 7 (http/https) concerns. For example, if you want to have a URL scheme where /service1 and /service2 go to separate Services, an Ingress controller can handle this for you. Similarly, an Ingress can be responsible for terminating TLS traffic, so that you don't have to configure it in each app.
The resulting traffic map is "public -> ELB (created by Service) -> Ingress -> Service -> Pod (running my app)". This can be a bit confusing, and Kubernetes is heading towards shifting responsibility for ELBs and similar from Services to Ingresses in a future version.
Just do it. Minikube is a tool to give you a toy cluster, running in a VM on your laptop.
brew cask install virtualbox
brew cask install minikube
brew install kubectl
minikube start
minikube dashboardThe guestbook tutorial will walk you through installing a full sample application.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
👏