K8s NodeSelector usage

gistfile1.txt

# List running nodes
docker run -it --rm --volumes-from data levenson/kubectl get nodes
NAME           LABELS                                            STATUS
10.161.34.62   kubernetes.io/hostname=10.161.34.62               Ready
10.161.34.65   kubernetes.io/hostname=10.161.34.65               Ready
10.161.34.70   kubernetes.io/hostname=10.161.34.70               Ready


# Label nodes with name key
docker run -it --rm --volumes-from data levenson/kubectl label nodes 10.161.34.62 name=node-1
docker run -it --rm --volumes-from data levenson/kubectl label nodes 10.161.34.65 name=node-2
docker run -it --rm --volumes-from data levenson/kubectl label nodes 10.161.34.70 name=node-3

# Confirm labels applied to nodes
docker run -it --rm --volumes-from data levenson/kubectl get nodes
NAME           LABELS                                            STATUS
10.161.34.62   kubernetes.io/hostname=10.161.34.62,name=node-1   Ready
10.161.34.65   kubernetes.io/hostname=10.161.34.65,name=node-2   Ready
10.161.34.70   kubernetes.io/hostname=10.161.34.70,name=node-3   Ready

# Create kafka-1 list,svc,pod.
docker run -it --rm --volumes-from data levenson/kubectl create -f /data/kafka-1.json

{
  "kind": "List",
  "apiVersion": "v1",
  "id": "kafka",
  "items":[
    {
      "kind": "Service",
      "apiVersion": "v1",
      "metadata": {
        "name": "kafka-1",
        "namespace": "realtime",
        "labels": {
          "name": "kafa-1"
        }
      },
      "spec": {
        "ports": [
          {
            "name": "service",
            "port": 9092,
            "targetPort": 9092
          },
          {
            "name": "jmx",
            "port": 7203,
            "targetPort": 7203
          }
        ],
        "selector": {
          "server-id": "1"
        }
      }
    },
    {
      "kind": "Pod",
      "apiVersion": "v1",
      "metadata": {
        "name": "kafka-1",
        "namespace": "realtime",
        "labels": {
          "name": "kafka",
          "server-id": "1"
        }
      },
      "spec": {
        "containers": [
          {
            "name": "server",
            "image": "levenson/k8s-kafka",
            "env":[
              { "name": "KAFKA_SERVER_ID", "value": "1" },
              { "name": "KAFKA_CLIENT_1_SERVICE_HOST", "value": "kafka-1.realtime.svc.kubernetes.io" },
              { "name": "KAFKA_CLIENT_1_SERVICE_PORT", "value": "9093" },
              { "name": "ZK_CLIENT_1_SERVICE_HOST", "value": "zookeeper.realtime.svc.kubernetes.io" },
              { "name": "ZK_CLIENT_1_SERVICE_PORT", "value": "2181" }
            ],
            "volumeMounts": [
              {
                "name": "kafka-1-data",
                "mountPath": "/data/"
              }
            ],
            "ports":[
              {
                "containerPort": 9092
              },
              {
                "containerPort": 7203
              }
            ]
          }
        ],
        "nodeSelector": {
          "name": "node-1"
        },
        "volumes": [
          {
            "name": "kafka-1-data",
            "hostPath": {
              "path": "/data/kafka-1/data"
            }
          }
        ]
      }
    }
  ]
}

# Confirm kafka-1 is running on node with label name=node-1 -- No it's running on node labelled name=node-2
docker run -it --rm --volumes-from data levenson/kubectl get pods kafka-1 -o wide --namespace=realtime
NAME      READY     STATUS    RESTARTS   AGE       NODE
kafka-1   1/1       Running   0          1h        10.161.34.65

Updated nodeSelector from node to name key and recreated pod. Same issue.

docker run -it --rm --volumes-from data levenson/kubectl get pods kafka-1 -o wide --namespace=realtime
NAME READY STATUS RESTARTS AGE NODE
kafka-1 0/1 Running 0 9s 10.161.34.65

After moving nodeSelector under the Pod spec rather than the container spec

docker run -it --rm --volumes-from data levenson/kubectl --validate create -f /data/kafka-1.json
error validating data: couldn't find type: v1.List

docker run -it --rm --volumes-from data levenson/kubectl create -f /data/kafka-1.json
services/kafka-1
pods/kafka-1

docker run -it --rm --volumes-from data levenson/kubectl get pods kafka-1 -o wide --namespace=realtime -o wide
NAME      READY     STATUS    RESTARTS   AGE       NODE
kafka-1   0/1       Running   0          6s        10.161.34.65

Found the issue was in the context placement of the nodeSelector key. I isolated using --validate. This is now working as expected.

@lachie83 awesome this is great news. Do the docs need to be updated?