Skip to content

Instantly share code, notes, and snippets.

@shpwrck

shpwrck/README.md

Last active June 9, 2023 16:35
Show Gist options
  • Save shpwrck/9d379c453e74dcf815b298fb27cdcbd6 to your computer and use it in GitHub Desktop.
Save shpwrck/9d379c453e74dcf815b298fb27cdcbd6 to your computer and use it in GitHub Desktop.
Download ZIP
Label Based Auth
Raw
README.md

Requirements

Provide a mechanism to enforce network security across clusters where membership is defined through the use of labels.

Components

Pre-Installed Components

  • Gloo Platform Control Plane Cluster, Gloo Agent Cluster A,B
    • Istio Deployment on Cluster A,B
    • Shared Trust (Root Trust Policy)

Included Components

  • Management Cluster
    • Workspaces & Workspace Settings << Allow for network segmentation
    • Namespaces << Store Mesh-Wide Config
    • Gateway Lifecycle Manager << Install East West Gateways
    • Virtual Destinations << Create Virtual Services for Cross Cluster Traffic
  • Workload Clusters
    • HTTPBIN Application << Application Deployments/Services/Namespaces
    • Gateway Namespaces and Services << Provide Namespaces with Labels for workspace inclusion
  • Cluster B
    • Network Policy << Example for Red Workspace

Explanation

Network Policy and East West Gateway Deployments can work in conjuction to secure the network even when spread across separate clusters. In this example a namespace is added to a workspace via a label:

  • workspace=red
  • workspace=blue
  • workspace=green

This implies that namespaces of different names can be joined together into a single workspace.

In each workspace a corresponding EastWest gateway is deployed. This allows a destination cluster (Cluster B) to apply network policy using the same labels as the workspace to control traffic that goes from one cluster to another.

In the diagram you can see each cluster and the three workspaces.

The information presented in the diagram is explained by the following table:

Traffic Endpoints Discovery Resource Enforcement Resource Notes
Green Namespace / Cluster A -> Red Namespace / Cluster A Kubernetes - Visible Network Policy - Namespace Selector DNS will resolve, but traffic will be dropped
Blue Namespace / Cluster A -> Red Namespace / Cluster A Kubernetes - Visible Network Policy - Namespace Selector DNS will resolve, but traffic will be dropped
Red Namespace / Cluster A -> Red Namespace / Cluster B Istio - Visible Network Policy - IP Block DNS will resolve, traffic will be forwarded
Blue Namespace / Cluster A -> Red Namespace / Cluster B Istio - Invisible Network Policy - IP Block DNS will not resolve, traffic will be dropped (if on separate node)
Green Namespace / Cluster A -> Red Namespace / Cluster B Istio - Invisible Network Policy - IP Block DNS will not resolve, traffic will be dropped (if on separate node)
Istio Namespace / Cluster A -> Any Namespace / Cluster A Kubernetes - Visible Network Policy - Namespace Selector DNS will resolve, traffic will be forwarded

The responsibilities of Istio are to provide DNS resolution and add additional layer 7 authorization as well as mTLS. The responsibilities of Kubernetes and Network Policy are to establish IP network boundaries using namespaces and labels.

In addition to what is provided here, an egress gateway could also be installed on the source cluster (Cluster A) to allow for a predefined IP to associate with the source cluster when defining IP Blocks in Network Policy.

Display the source blob
Display the rendered blob
Raw
diagram.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Raw
glm.yaml
apiVersion: admin.gloo.solo.io/v2
kind: GatewayLifecycleManager
metadata:
name: perworkspace-ew
namespace: gloo-mesh
spec:
installations:
- clusters:
- activeGateway: false
name: cluster1
- activeGateway: false
name: cluster2
gatewayRevision: 1-16-red
controlPlaneRevision: 1-16
istioOperatorSpec:
components:
ingressGateways:
- enabled: true
k8s:
env:
- name: ISTIO_META_ROUTER_MODE
value: sni-dnat
service:
ports:
- name: status-port
port: 15021
targetPort: 15021
- name: tls
nodePort: 31313
port: 15443
targetPort: 15443
selector:
istio: eastwestgateway
type: LoadBalancer
label:
app: istio-eastwestgateway
istio: eastwestgateway
revision: 1-16
name: istio-eastwestgateway
namespace: red-istio-system
hub: us-docker.pkg.dev/gloo-mesh/istio-7a97385594af
profile: empty
tag: 1.16.2-solo
values:
gateways:
istio-ingressgateway:
injectionTemplate: gateway
- clusters:
- activeGateway: false
name: cluster1
- activeGateway: false
name: cluster2
gatewayRevision: 1-16-blue
controlPlaneRevision: 1-16
istioOperatorSpec:
components:
ingressGateways:
- enabled: true
k8s:
env:
- name: ISTIO_META_ROUTER_MODE
value: sni-dnat
service:
ports:
- name: status-port
port: 15021
targetPort: 15021
- name: tls
nodePort: 31313
port: 15443
targetPort: 15443
selector:
istio: eastwestgateway
type: LoadBalancer
label:
app: istio-eastwestgateway
istio: eastwestgateway
revision: 1-16
name: istio-eastwestgateway
namespace: blue-istio-system
hub: us-docker.pkg.dev/gloo-mesh/istio-7a97385594af
profile: empty
tag: 1.16.2-solo
values:
gateways:
istio-ingressgateway:
injectionTemplate: gateway
- clusters:
- activeGateway: false
name: cluster1
- activeGateway: false
name: cluster2
gatewayRevision: 1-16-green
controlPlaneRevision: 1-16
istioOperatorSpec:
components:
ingressGateways:
- enabled: true
k8s:
env:
- name: ISTIO_META_ROUTER_MODE
value: sni-dnat
service:
ports:
- name: status-port
port: 15021
targetPort: 15021
- name: tls
nodePort: 31313
port: 15443
targetPort: 15443
selector:
istio: eastwestgateway
type: LoadBalancer
label:
app: istio-eastwestgateway
istio: eastwestgateway
revision: 1-16
name: istio-eastwestgateway
namespace: green-istio-system
hub: us-docker.pkg.dev/gloo-mesh/istio-7a97385594af
profile: empty
tag: 1.16.2-solo
values:
gateways:
istio-ingressgateway:
injectionTemplate: gateway
Raw
httpbin-blue.yaml
---
apiVersion: v1
kind: Namespace
metadata:
labels:
workspace: blue
istio.io/rev: 1-16
name: blue-httpbin
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: httpbin
namespace: blue-httpbin
---
apiVersion: v1
kind: Service
metadata:
name: httpbin
namespace: blue-httpbin
labels:
app: httpbin
service: httpbin
federation: enabled
spec:
ports:
- name: http
port: 8000
targetPort: 80
selector:
app: httpbin
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: httpbin
namespace: blue-httpbin
spec:
replicas: 1
selector:
matchLabels:
app: httpbin
version: v1
template:
metadata:
labels:
app: httpbin
version: v1
spec:
serviceAccountName: httpbin
containers:
- image: docker.io/kong/httpbin
imagePullPolicy: IfNotPresent
name: httpbin
ports:
- containerPort: 80
Raw
httpbin-green.yaml
---
apiVersion: v1
kind: Namespace
metadata:
labels:
workspace: green
istio.io/rev: 1-16
name: green-httpbin
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: httpbin
namespace: green-httpbin
---
apiVersion: v1
kind: Service
metadata:
name: httpbin
namespace: green-httpbin
labels:
app: httpbin
service: httpbin
federation: enabled
spec:
ports:
- name: http
port: 8000
targetPort: 80
selector:
app: httpbin
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: httpbin
namespace: green-httpbin
spec:
replicas: 1
selector:
matchLabels:
app: httpbin
version: v1
template:
metadata:
labels:
app: httpbin
version: v1
spec:
serviceAccountName: httpbin
containers:
- image: docker.io/kong/httpbin
imagePullPolicy: IfNotPresent
name: httpbin
ports:
- containerPort: 80
Raw
httpbin-red.yaml
---
apiVersion: v1
kind: Namespace
metadata:
labels:
workspace: red
istio.io/rev: 1-16
name: red-test
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: httpbin
namespace: red-test
---
apiVersion: v1
kind: Service
metadata:
name: httpbin
namespace: red-test
labels:
app: httpbin
service: httpbin
federation: enabled
spec:
ports:
- name: http
port: 8000
targetPort: 80
selector:
app: httpbin
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: httpbin
namespace: red-test
spec:
replicas: 1
selector:
matchLabels:
app: httpbin
version: v1
template:
metadata:
labels:
app: httpbin
version: v1
spec:
serviceAccountName: httpbin
containers:
- image: docker.io/kong/httpbin
imagePullPolicy: IfNotPresent
name: httpbin
ports:
- containerPort: 80
Raw
namespaces.yaml
---
apiVersion: v1
kind: Namespace
metadata:
labels:
workspace: red
name: red-config
---
apiVersion: v1
kind: Namespace
metadata:
labels:
workspace: green
name: green-config
---
apiVersion: v1
kind: Namespace
metadata:
labels:
workspace: blue
name: blue-config
Raw
network-policy.yaml
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: default-deny-ingress
namespace: red-httpbin
spec:
podSelector: {}
policyTypes:
- Ingress
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: allow-matching-red-labels
namespace: red-httpbin
spec:
podSelector: {}
policyTypes:
- Ingress
ingress:
- from:
- namespaceSelector:
matchLabels:
workspace: red
- namespaceSelector:
matchLabels:
workspace: ${ISTIO_CONTROL_PLANE}
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: default-deny-ingress
namespace: red-istio-system
spec:
podSelector: {}
policyTypes:
- Ingress
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: allow-matching-red-labels
namespace: red-istio-system
spec:
podSelector: {}
policyTypes:
- Ingress
ingress:
- from:
- namespaceSelector:
matchLabels:
workspace: red
- namespaceSelector:
matchLabels:
workspace: ${ISTIO_CONTROL_PLANE}
- ipBlock:
cidr: ${SOURCE_IP}
Raw
vd.yaml
---
apiVersion: networking.gloo.solo.io/v2
kind: VirtualDestination
metadata:
name: red-httpbin
namespace: red-config
spec:
hosts:
- red-httpbin.mesh
ports:
- number: 8000
protocol: HTTP
services:
- name: httpbin
namespace: red-httpbin
cluster: cluster2
---
apiVersion: networking.gloo.solo.io/v2
kind: VirtualDestination
metadata:
name: blue-httpbin
namespace: blue-config
spec:
hosts:
- blue-httpbin.mesh
ports:
- number: 8000
protocol: HTTP
services:
- name: httpbin
namespace: blue-httpbin
cluster: cluster2
---
apiVersion: networking.gloo.solo.io/v2
kind: VirtualDestination
metadata:
name: green-httpbin
namespace: green-config
spec:
hosts:
- green-httpbin.mesh
ports:
- number: 8000
protocol: HTTP
services:
- name: httpbin
namespace: green-httpbin
cluster: cluster2
Raw
work-ns.yaml
---
apiVersion: v1
kind: Namespace
metadata:
labels:
workspace: blue
name: blue-istio-system
---
apiVersion: v1
kind: Namespace
metadata:
labels:
workspace: red
name: red-istio-system
---
apiVersion: v1
kind: Namespace
metadata:
labels:
workspace: red
name: red-test
---
apiVersion: v1
kind: Namespace
metadata:
labels:
workspace: green
name: green-istio-system
Raw
work-svc.yaml
---
apiVersion: v1
kind: Service
metadata:
name: istio-eastwestgateway-1-16
namespace: red-istio-system
spec:
externalTrafficPolicy: Cluster
internalTrafficPolicy: Cluster
ipFamilies:
- IPv4
ipFamilyPolicy: SingleStack
ports:
- name: status-port
port: 15021
protocol: TCP
targetPort: 15021
- name: tls
port: 15443
protocol: TCP
targetPort: 15443
selector:
app: istio-eastwestgateway
istio: eastwestgateway
revision: 1-16
sessionAffinity: None
type: LoadBalancer
---
apiVersion: v1
kind: Service
metadata:
name: istio-eastwestgateway-1-16
namespace: blue-istio-system
spec:
externalTrafficPolicy: Cluster
internalTrafficPolicy: Cluster
ipFamilies:
- IPv4
ipFamilyPolicy: SingleStack
ports:
- name: status-port
port: 15021
protocol: TCP
targetPort: 15021
- name: tls
port: 15443
protocol: TCP
targetPort: 15443
selector:
app: istio-eastwestgateway
istio: eastwestgateway
revision: 1-16
sessionAffinity: None
type: LoadBalancer
---
apiVersion: v1
kind: Service
metadata:
name: istio-eastwestgateway-1-16
namespace: green-istio-system
spec:
externalTrafficPolicy: Cluster
internalTrafficPolicy: Cluster
ipFamilies:
- IPv4
ipFamilyPolicy: SingleStack
ports:
- name: status-port
port: 15021
protocol: TCP
targetPort: 15021
- name: tls
port: 15443
protocol: TCP
targetPort: 15443
selector:
app: istio-eastwestgateway
istio: eastwestgateway
revision: 1-16
sessionAffinity: None
type: LoadBalancer
Raw
workspaces.yaml
---
apiVersion: admin.gloo.solo.io/v2
kind: Workspace
metadata:
name: red
namespace: gloo-mesh
spec:
workloadClusters:
- name: mgmt-cluster
namespaces:
- labels:
workspace: red
- name: cluster*
namespaces:
- labels:
workspace: red
---
apiVersion: admin.gloo.solo.io/v2
kind: Workspace
metadata:
name: blue
namespace: gloo-mesh
spec:
workloadClusters:
- name: mgmt-cluster
namespaces:
- labels:
workspace: blue
- name: cluster*
namespaces:
- labels:
workspace: blue
---
apiVersion: admin.gloo.solo.io/v2
kind: Workspace
metadata:
name: green
namespace: gloo-mesh
spec:
workloadClusters:
- name: mgmt-cluster
namespaces:
- labels:
workspace: green
- name: cluster*
namespaces:
- labels:
workspace: green
Raw
workspacesettings.yaml
---
apiVersion: admin.gloo.solo.io/v2
kind: WorkspaceSettings
metadata:
name: red
namespace: red-config
spec:
exportTo:
- workspaces:
- name: not-migrated
importFrom:
- workspaces:
- name: not-migrated
options:
eastWestGateways:
- selector:
workspace: red
federation:
enabled: false
serviceSelector:
- labels:
federation: enabled
serviceIsolation:
enabled: false
trimProxyConfig: false
trimAllProxyConfig: false
---
apiVersion: admin.gloo.solo.io/v2
kind: WorkspaceSettings
metadata:
name: green
namespace: green-config
spec:
exportTo:
- workspaces:
- name: not-migrated
importFrom:
- workspaces:
- name: not-migrated
options:
eastWestGateways:
- selector:
workspace: green
federation:
enabled: false
serviceSelector:
- labels:
federation: enabled
serviceIsolation:
enabled: false
trimProxyConfig: false
trimAllProxyConfig: false
---
apiVersion: admin.gloo.solo.io/v2
kind: WorkspaceSettings
metadata:
name: blue
namespace: blue-config
spec:
exportTo:
- workspaces:
- name: not-migrated
importFrom:
- workspaces:
- name: not-migrated
options:
eastWestGateways:
- selector:
workspace: blue
federation:
enabled: false
serviceSelector:
- labels:
federation: enabled
serviceIsolation:
enabled: false
trimProxyConfig: false
trimAllProxyConfig: false
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment