Skip to content

Instantly share code, notes, and snippets.

@miabbott
Last active December 11, 2015 19:14
Show Gist options
  • Star 2 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save miabbott/83718f5020a5c31de1ca to your computer and use it in GitHub Desktop.
Save miabbott/83718f5020a5c31de1ca to your computer and use it in GitHub Desktop.
Running Containerized Kubernetes on RHELAH 7.2 (Single Node)
  1. Configure etcd
  2. Enable + start etcd service
  3. Configure flanneld
  4. Enable + Start flanneld service
  5. Reboot
  6. Pull down kubernetes containers
  7. Configure kubernetes manifests
  8. Configure kubernetes services
  9. Configure kubelet service
  10. Enable + Start kubelet/kube-proxy service

Notes

  • Because these instructions assume we are setting up kubernetes in a single node environment, when you see the $MASTER_IP variable, you can use the IP address of the host you are working with.
  • As of RHELAH 7.2.1, the kubelet has not been containerized, so we will use the on host kubelet service.
  • The current recommended way to use the manifests is to have /etc/kubernetes mounted from the host into the containers and the services will use the config files in that directory.

Configure etcd

Edit /etc/etcd/etcd.conf

ETCD_NAME=default
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_LISTEN_CLIENT_URLS="http://0.0.0.0:2379"
ETCD_LISTEN_PEER_URLS="http://localhost:2380"
ETCD_ADVERTISE_CLIENT_URLS="http://0.0.0.0:2379"

Enable + Start etcd service

# systemctl enable etcd
# systemctl start etcd

Configure flanneld

Make sure /etc/sysconfig/flanneld has the same value for FLANNEL_ETCD_KEY that will be used in this step. In this case, we have a value of /atomic.io/network

# grep FLANNEL_ETCD_KEY /etc/sysconfig/flanneld
FLANNEL_ETCD_KEY="/atomic.io/network"

Create /root/flannel-config.json

NOTE: The value of "Network" below must not conflict with any existing IP networks on the host.

{
  "Network": "10.253.0.0/16",
  "SubnetLen": 24,
  "Backend": {
    "Type": "vxlan",
    "VNI": 1
  }
}

Load in the flannel-config.json to etcd

# etcdctl set atomic.io/network/config < /root/flannel-config.json

Enable + Start flanneld service

# systemctl enable flanneld
# systemctl start flanneld

Reboot

NOTE: A reboot is required for the docker service to re-enumerate the docker0 interface with an IP address from the flannel network.

# systemctl reboot

Pull down kubernetes containers

# docker pull registry.access.redhat.com/rhel7/kubernetes-apiserver
# docker pull registry.access.redhat.com/rhel7/kubernetes-controller-mgr
# docker pull registry.access.redhat.com/rhel7/kubernetes-scheduler

Configure kubernetes manifests

Make the manifests directory

# mkdir -p /etc/kubernetes/manifests/

Create the following files using the content in the Manifests section at the end of this document:

  • /etc/kubernetes/manifests/apiserver-pod.json
  • /etc/kubernetes/manifests/controller-mgr-pod.json
  • /etc/kubernetes/manifests/scheduler-pod.json

Configure kubernetes services

Edit /etc/kubernetes/config

NOTE: We have to specify the --address=0.0.0.0 option somewhere, so that the kubernetes services are available on the host IP of the system. This allows the health checks to pass when queried on the host IP. There is no great place to put this, so I settled on the KUBE_MASTER option.

KUBE_LOGTOSTDERR="--logtostderr=true"
KUBE_LOG_LEVEL="--v=0"
KUBE_ALLOW_PRIV="--allow-privileged=false"
KUBE_MASTER="--master=http://$MASTER_IP:8080 --address=0.0.0.0"

Edit /etc/kubernetes/apiserver

KUBE_API_ADDRESS="--insecure-bind-address=0.0.0.0"
KUBE_ETCD_SERVERS="--etcd_servers=http://127.0.0.1:2379"
KUBE_SERVICE_ADDRESSES="--service-cluster-ip-range=10.254.0.0/16"
KUBE_ADMISSION_CONTROL="--admission_control=NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ResourceQuota"

Configure kubelet service

NOTE: As of RHELAH 7.2.1, we are still using the on host kubelet service to start up the kubernetes services.

Edit /etc/kubernetest/kubelet

KUBELET_ADDRESS="--address=0.0.0.0"
KUBELET_HOSTNAME="--hostname-override=$MASTER_IP"
KUBELET_ARGS="--register-node=true --config=/etc/kubernetes/manifests/"
KUBELET_API_SERVER="--api_servers=http://$MASTER_IP:8080"

Enable + Start kubelet service + kube-proxy service

# systemctl enable kube-proxy kubelet
# systemctl start kube-proxy kubelet

Inspect kubernetes

# kubectl --server=$MASTER_IP:8080 get nodes
# kubectl --server=$MASTER_IP:8080 get svc --all-namespaces=true
# kubectl --server=$MASTER_IP:8080 get po --all-namespaces=true
# docker ps

Example invocation and output

-bash-4.2# kubectl --server=172.16.69.53:8080 get nodes
NAME           LABELS                                STATUS
172.16.69.53   kubernetes.io/hostname=172.16.69.53   Ready
-bash-4.2# kubectl --server=172.16.69.53:8080 get svc --all-namespaces=true
NAMESPACE   NAME         LABELS                                    SELECTOR   IP(S)        PORT(S)
default     kubernetes   component=apiserver,provider=kubernetes   <none>     10.254.0.1   443/TCP
-bash-4.2# kubectl --server=172.16.69.53:8080 get pods --all-namespaces=true
NAMESPACE   NAME                                   READY     STATUS    RESTARTS   AGE
default     kube-apiserver-172.16.69.53            1/1       Running   0          46s
default     kube-controller-manager-172.16.69.53   1/1       Running   0          45s
default     kube-scheduler-172.16.69.53            1/1       Running   0          45s
-bash-4.2# docker ps
CONTAINER ID        IMAGE                                  COMMAND                  CREATED              STATUS              PORTS               NAMES
c7db631efdb5        rhel7/kubernetes-controller-mgr        "/usr/bin/kube-contro"   About a minute ago   Up About a minute                       k8s_kube-controller-manager.3e0b7d06_kube-controller-manager-172.16.69.53_default_da1c3e6d2440aba1b991528bd29dde66_1f6081b2
5fe503609061        rhel7/kubernetes-scheduler             "/usr/bin/kube-schedu"   About a minute ago   Up About a minute                       k8s_kube-scheduler.13365e34_kube-scheduler-172.16.69.53_default_e26abdc8f6ca98a67c57fbd347564cd9_3590ae9b
e764dccc77bb        gcr.io/google_containers/pause:0.8.0   "/pause"                 About a minute ago   Up About a minute                       k8s_POD.e4cc795_kube-controller-manager-172.16.69.53_default_da1c3e6d2440aba1b991528bd29dde66_0d021f0a
f5b307ae1938        gcr.io/google_containers/pause:0.8.0   "/pause"                 About a minute ago   Up About a minute                       k8s_POD.e4cc795_kube-scheduler-172.16.69.53_default_e26abdc8f6ca98a67c57fbd347564cd9_a65fe23a
07f93349f5be        rhel7/kubernetes-apiserver             "/usr/bin/kube-apiser"   About a minute ago   Up About a minute                       k8s_kube-apiserver.26c7bffc_kube-apiserver-172.16.69.53_default_b9c5297e12b6464b9ffdaa38152c73c2_2e95b374
17df0912aa79        gcr.io/google_containers/pause:0.8.0   "/pause"                 About a minute ago   Up About a minute                       k8s_POD.e4cc795_kube-apiserver-172.16.69.53_default_b9c5297e12b6464b9ffdaa38152c73c2_5d6718a9

Manifests

apiserver-pod.json

{
  "kind": "Pod",
  "apiVersion": "v1",
  "metadata": {
    "name": "kube-apiserver"
  },
  "spec": {
    "hostNetwork": true,
    "containers": [
      {
        "name": "kube-apiserver",
        "image": "rhel7/kubernetes-apiserver",
        "ports": [
          {
            "name": "https",
            "hostPort": 443,
            "containerPort": 443
          },
          {
            "name": "local",
            "hostPort": 8080,
            "containerPort": 8080
          }
        ],
        "volumeMounts": [
          {
            "name": "etcssl",
            "mountPath": "/etc/ssl",
            "readOnly": true
          },
          {
            "name": "config",
            "mountPath": "/etc/kubernetes",
            "readOnly": true
          }
        ],
        "livenessProbe": {
          "httpGet": {
            "path": "/healthz",
            "port": 8080
          },
          "initialDelaySeconds": 15,
          "timeoutSeconds": 15
        }
      }
    ],
    "volumes": [
      {
        "name": "etcssl",
        "hostPath": {
          "path": "/etc/ssl"
        }
      },
      {
        "name": "config",
        "hostPath": {
          "path": "/etc/kubernetes"
        }
      }
    ]
  }
}

controller-mgr-pod.json

{
  "kind": "Pod",
  "apiVersion": "v1",
  "metadata": {
    "name": "kube-controller-manager"
  },
  "spec": {
    "hostNetwork": true,
    "containers": [
      {
        "name": "kube-controller-manager",
        "image": "rhel7/kubernetes-controller-mgr",
        "volumeMounts": [
          {
            "name": "etcssl",
            "mountPath": "/etc/ssl",
            "readOnly": true
          },
          {
            "name": "config",
            "mountPath": "/etc/kubernetes",
            "readOnly": true
          }
        ],
        "livenessProbe": {
          "httpGet": {
            "path": "/healthz",
            "port": 10252
          },
          "initialDelaySeconds": 15,
          "timeoutSeconds": 15
        }
      }
    ],
    "volumes": [
      {
        "name": "etcssl",
        "hostPath": {
          "path": "/etc/ssl"
        }
      },
      {
        "name": "config",
        "hostPath": {
          "path": "/etc/kubernetes"
        }
      }
    ]
  }
}

scheduler-pod.json

{
  "kind": "Pod",
  "apiVersion": "v1",
  "metadata": {
    "name": "kube-scheduler"
  },
  "spec": {
    "hostNetwork": true,
    "containers": [
      {
        "name": "kube-scheduler",
        "image": "rhel7/kubernetes-scheduler",
        "volumeMounts": [
          {
            "name": "config",
            "mountPath": "/etc/kubernetes",
            "readOnly": true
          }
        ],
        "livenessProbe": {
          "httpGet": {
            "path": "/healthz",
            "port": 10251
          },
          "initialDelaySeconds": 15,
          "timeoutSeconds": 15
        }
      }
    ],
    "volumes": [
      {
        "name": "config",
        "hostPath": {
          "path": "/etc/kubernetes"
        }
      }
    ]
  }
}

References

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment