Skip to content

Instantly share code, notes, and snippets.

@dionysius
Last active July 30, 2019 16:26
Show Gist options
  • Save dionysius/c0897e7eb980eb89931e185b2644fe57 to your computer and use it in GitHub Desktop.
Save dionysius/c0897e7eb980eb89931e185b2644fe57 to your computer and use it in GitHub Desktop.
kubefed v2 federate resource only to member clusters
#I have the following contexts
$ kubectl config get-contexts
CURRENT NAME CLUSTER AUTHINFO NAMESPACE
* cluster0 cluster0 cluster0
cluster1 cluster1 cluster1
cluster2 cluster2 cluster2
#Using cluster0, as there is kubefed installed
$ kubectl config use-context cluster0
Switched to context "cluster0".
#Joining the clusters 1 and 2 to 0
$ kubefedctl join cluster1 --cluster-context cluster1 --host-cluster-context cluster0 --v=2
I0729 16:39:00.708690 16966 join.go:159] Args and flags: name cluster1, host: cluster0, host-system-namespace: kube-federation-system, kubeconfig: , cluster-context: cluster1, secret-name: , dry-run: false
I0729 16:39:01.271447 16966 join.go:219] Performing preflight checks.
I0729 16:39:01.319467 16966 join.go:225] Creating kube-federation-system namespace in joining cluster
I0729 16:39:01.356406 16966 join.go:233] Created kube-federation-system namespace in joining cluster
I0729 16:39:01.356471 16966 join.go:236] Creating cluster credentials secret
I0729 16:39:01.356500 16966 join.go:372] Creating service account in joining cluster: cluster1
I0729 16:39:01.428578 16966 join.go:382] Created service account: cluster1-cluster0 in joining cluster: cluster1
I0729 16:39:01.428633 16966 join.go:410] Creating cluster role and binding for service account: cluster1-cluster0 in joining cluster: cluster1
I0729 16:39:01.530790 16966 join.go:419] Created cluster role and binding for service account: cluster1-cluster0 in joining cluster: cluster1
I0729 16:39:01.530817 16966 join.go:423] Creating secret in host cluster: cluster0
I0729 16:39:02.564661 16966 join.go:812] Using secret named: cluster1-cluster0-token-jtwpt
I0729 16:39:02.598832 16966 join.go:855] Created secret in host cluster named: cluster1-mzkxr
I0729 16:39:02.598862 16966 join.go:432] Created secret in host cluster: cluster0
I0729 16:39:02.598881 16966 join.go:246] Cluster credentials secret created
I0729 16:39:02.598900 16966 join.go:248] Creating federated cluster resource
I0729 16:39:02.683737 16966 join.go:257] Created federated cluster resource
$ kubefedctl join cluster2 --cluster-context cluster2 --host-cluster-context cluster0 --v=2
I0729 16:39:23.423549 17008 join.go:159] Args and flags: name cluster2, host: cluster0, host-system-namespace: kube-federation-system, kubeconfig: , cluster-context: cluster2, secret-name: , dry-run: false
I0729 16:39:24.076561 17008 join.go:219] Performing preflight checks.
I0729 16:39:24.175559 17008 join.go:225] Creating kube-federation-system namespace in joining cluster
I0729 16:39:26.518549 17008 join.go:233] Created kube-federation-system namespace in joining cluster
I0729 16:39:26.518603 17008 join.go:236] Creating cluster credentials secret
I0729 16:39:26.518627 17008 join.go:372] Creating service account in joining cluster: cluster2
I0729 16:39:28.691224 17008 join.go:382] Created service account: cluster2-cluster0 in joining cluster: cluster2
I0729 16:39:28.691246 17008 join.go:410] Creating cluster role and binding for service account: cluster2-cluster0 in joining cluster: cluster2
I0729 16:39:30.129655 17008 join.go:419] Created cluster role and binding for service account: cluster2-cluster0 in joining cluster: cluster2
I0729 16:39:30.129729 17008 join.go:423] Creating secret in host cluster: cluster0
I0729 16:39:34.169334 17008 join.go:812] Using secret named: cluster2-cluster0-token-bkzcc
I0729 16:39:34.208201 17008 join.go:855] Created secret in host cluster named: cluster2-p86lw
I0729 16:39:34.208230 17008 join.go:432] Created secret in host cluster: cluster0
I0729 16:39:34.208247 17008 join.go:246] Cluster credentials secret created
I0729 16:39:34.208263 17008 join.go:248] Creating federated cluster resource
I0729 16:39:34.373199 17008 join.go:257] Created federated cluster resource
#The namespace I want it's content to be federated
$ kubectl create ns test-namespace
namespace/test-namespace created
$ kubefedctl federate namespace test-namespace --contents
I0729 16:44:46.832530 17989 federate.go:459] Resource to federate is a namespace. Given namespace will itself be the container for the federated namespace
I0729 16:44:46.871545 17989 federate.go:488] Successfully created FederatedNamespace "test-namespace/test-namespace" from Namespace
I0729 16:44:46.905443 17989 federate.go:488] Successfully created FederatedSecret "test-namespace/default-token-tsks4" from Secret
I0729 16:44:46.918981 17989 federate.go:488] Successfully created FederatedServiceAccount "test-namespace/default" from ServiceAccount
#The following clusters are used for the namespace federation
$ kubectl get federatednamespaces.types.kubefed.k8s.io -n test-namespace test-namespace -o json | jq -r '.status.clusters[].name'
cluster2
cluster1
#I have the following deployment, I want only to run on cluster1 and cluster2
$ cat test-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
namespace: test-namespace
spec:
replicas: 1
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx:1.7.9
ports:
- containerPort: 80
#Creating that deployment
$ kubectl create -f test-deployment.yaml
deployment.apps/nginx-deployment created
#Federate that deployment
$ kubefedctl federate deployments.apps -n test-namespace nginx-deployment --host-cluster-context=cluster0
I0730 18:08:22.883205 25504 federate.go:488] Successfully created FederatedDeployment "test-namespace/nginx-deployment" from Deployment
#But the pods are created also on cluster0 (we're still in context cluster0)
$ kubectl -n test-namespace get pods
NAME READY STATUS RESTARTS AGE
nginx-deployment-5754944d6c-hhrjf 1/1 Running 0 14m
#Lets check the status of that federated deployment:
$ kubectl get federateddeployments.types.kubefed.k8s.io -n test-namespace nginx-deployment -o json | jq -r '.status'
{
"clusters": [
{
"name": "cluster2"
},
{
"name": "cluster1"
}
],
"conditions": [
{
"lastProbeTime": "2019-07-30T16:14:17Z",
"lastTransitionTime": "2019-07-30T16:06:11Z",
"status": "True",
"type": "Propagation"
}
]
}
#So far so good, the object got federated
$ for c in cluster0 cluster1 cluster2; do echo "###$c:###"; kubectl --context "$c" -n test-namespace get deployment; done
###cluster0:###
NAME READY UP-TO-DATE AVAILABLE AGE
nginx-deployment 1/1 1 1 25h
###cluster1:###
NAME READY UP-TO-DATE AVAILABLE AGE
nginx-deployment 0/1 1 0 2m24s
###cluster2:###
NAME READY UP-TO-DATE AVAILABLE AGE
nginx-deployment 0/1 1 0 2m25s
#So, my problem is, I'd like to prevent, that there is a pod on cluster0, how?
$ for c in cluster0 cluster1 cluster2; do echo "###$c:###"; kubectl --context "$c" -n test-namespace get pod; done
###cluster0:###
NAME READY STATUS RESTARTS AGE
nginx-deployment-5754944d6c-hhrjf 1/1 Running 0 25h
###cluster1:###
NAME READY STATUS RESTARTS AGE
nginx-deployment-5754944d6c-h6zc2 0/1 Pending 0 4m4s
###cluster2:###
NAME READY STATUS RESTARTS AGE
nginx-deployment-5754944d6c-2whn2 0/1 Pending 0 4m3s
#This is now a FederatedDeployment without a matching resource in cluster0
$ cat onlyfederated.yaml
apiVersion: types.kubefed.k8s.io/v1beta1
kind: FederatedDeployment
metadata:
name: onlyfederated
namespace: test-namespace
spec:
placement:
clusterSelector:
matchLabels: {}
template:
spec:
replicas: 1
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- image: nginx:1.7.9
imagePullPolicy: IfNotPresent
name: nginx
ports:
- containerPort: 80
protocol: TCP
#Create it
$ kubectl create -f onlyfederated.yaml
federateddeployment.types.kubefed.k8s.io/onlyfederated created
#And check where it got federated
$ for c in cluster0 cluster1 cluster2; do echo "###$c:###"; kubectl --context "$c" -n test-namespace get deployment; done
###cluster0:###
NAME READY UP-TO-DATE AVAILABLE AGE
nginx-deployment 1/1 1 1 25h
###cluster1:###
NAME READY UP-TO-DATE AVAILABLE AGE
nginx-deployment 0/1 1 0 13m
onlyfederated 0/1 1 0 27s
###cluster2:###
NAME READY UP-TO-DATE AVAILABLE AGE
nginx-deployment 1/1 1 1 13m
onlyfederated 1/1 1 1 28s
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment