Skip to content

Instantly share code, notes, and snippets.

@pwillis-els

pwillis-els/gist:2df85ee1658b8bf40c2a3d52392c081c

Last active November 22, 2021 01:35
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save pwillis-els/2df85ee1658b8bf40c2a3d52392c081c to your computer and use it in GitHub Desktop.
Save pwillis-els/2df85ee1658b8bf40c2a3d52392c081c to your computer and use it in GitHub Desktop.
Download ZIP
Kubernetes Multitenancy: November 2021
Raw
gistfile1.md

This is an overview of the different options for multi-tenancy in Kubernetes as of November 2021.

Kubernetes Multi-Tentancy SIG (https://github.com/kubernetes-sigs/multi-tenancy)

  • Multi-Tenancy Benchmarks (https://github.com/kubernetes-sigs/multi-tenancy/tree/master/benchmarks)
  • Hierarchical Namespace Controller (https://github.com/kubernetes-sigs/hierarchical-namespaces)
    • "You can create additional namespaces under your team's namespace, even if you don't have cluster-level permission to create namespaces, and easily apply policies like RBAC and Network Policies across all namespaces in your team"
  • VirtualCluster (https://github.com/kubernetes-sigs/cluster-api-provider-nested/tree/main/virtualcluster)
    • "each tenant is assigned a dedicated tenant control plane, which is a upstream Kubernetes distribution. Tenants can create cluster scope resources such as namespaces and CRDs in the tenant control plane without affecting others. As a result, most of the isolation problems due to sharing one apiserver disappear. The Kubernetes cluster that manages the actual physical nodes is called a super cluster, which now becomes a Pod resource provider"
    • Components:
      • "vc-manager manages the lifecycle of each VirtualCluster custom resource. Based on the specification, it either creates CAPN control plane Pods in local K8s cluster, or imports an existing cluster if a valid kubeconfig is provided."
      • syncer - Maps the difference between a tenant's view of the world, and the actual state of things on the Kubernetes "super master". Manipulates Pod templates, ensures data consistency, and makes it so that the user doesn't need to know anything other than their own tenant namespaces and other details.
      • "vn-agent - A node daemon that proxies all tenant kubelet API requests to the kubelet process that running in the node. It ensures each tenant can only access its own Pods in the node."

Capsule (https://github.com/clastix/capsule)

  • "Capsule helps to implement a multi-tenancy and policy-based environment in your Kubernetes cluster. It is not intended to be yet another PaaS, instead, it has been designed as a micro-services-based ecosystem with the minimalist approach, leveraging only on upstream Kubernetes."
  • "In a single cluster, the Capsule Controller aggregates multiple namespaces in a lightweight abstraction called Tenant, basically a grouping of Kubernetes Namespaces. Within each tenant, users are free to create their namespaces and share all the assigned resources while the Capsule Policy Engine keeps the different tenants isolated from each other. The Network and Security Policies, Resource Quota, Limit Ranges, RBAC, and other policies defined at the tenant level are automatically inherited by all the namespaces in the tenant. Then users are free to operate their tenants in autonomy, without the intervention of the cluster administrator."
  • "Assign to tenants a dedicated set of compute, storage, and network resources and avoid the noisy neighbors' effect."
  • https://github.com/clastix/capsule/blob/master/docs/content/operator/use-cases/overview.md

Loft (https://github.com/loft-sh/loft)

  • Closed source implementation of multiple namespaces and virtual clusters
  • On top of Kiosk, adds
    • Mutli-cluster access for granting access to spaces in different clusters
    • Sleep mode scales down deployments in a space during periods of inactivity
    • Single sign-on with OIDC authentication providers like GitHub
  • https://loft.sh/blog/introduction-into-virtual-clusters-in-kubernetes/

Kiosk (https://github.com/loft-sh/kiosk)

  • The open source component driving multi-tenancy in Loft

Virtual Kubelet (https://github.com/virtual-kubelet/virtual-kubelet)

Cons

  • compatibility issues
  • intended to extend Kubernetes API into serverless container platforms like ACI and Fargate
  • not intended to be an alternative to Kubernetes federation

Arktos (https://github.com/CentaurusInfra/arktos)

Cons

  • Shared control plane for all users
  • Requires APIServer to support new tenant-specific APIs
  • All existing plugins would need to be changed

K3v (https://github.com/ibuildthecloud/k3v)

Cons

  • Proof of concept
  • Not updated in 2 years

Notes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment