Skip to content

Instantly share code, notes, and snippets.

@danehans

danehans/00_docs.md

Last active March 17, 2020 16:45
Show Gist options
  • Save danehans/10b97d53f7942f9793f234177c0ecb62 to your computer and use it in GitHub Desktop.
Save danehans/10b97d53f7942f9793f234177c0ecb62 to your computer and use it in GitHub Desktop.
Download ZIP
service-apis virtual host concept explained
Raw
00_docs.md

Intro

Service API's provide name-based virtual hosting of objects that reside in a Kubernetes cluster.

Gateway

A Gateway host's one or more names that clients connect to. A Gateway exposes these names on one or more network endpoints called listeners. If applicable, the TLS configuration of the hosted name is used to perform a TLS handshake. The hosted name is used to match a VirtualHost.

Persona: Cluster Ops

VirtualHost

A VirtualHost is an in-cluster object, i.e. Service, exposed by a Gateway. A VirtualHost performs request manipulation (optional) and routing based on match, filter and action rules.

Persona: Dev

Connection Request Workflow

  1. A client makes a request: https://foo.example.com
  2. The client resolves "foo.example.com" to IP address 1.1.1.1
  3. The client requests a TLS handshake with 1.1.1.1 using "foo.example.com" as the SNI server_name.
  4. The request is received on listener "my-tls-listener" of a controller configured by GatewayClass "acme-tls-lb" and Gateway "my-tls-gateway".
  5. The controller verifies that "foo.example.com" matches a name in gateway.spec.hostedNames[].
  6. A match is found, so the client's connection request is processed.
  7. The controller uses the TLS config associated to gateway.spec.hostedName["foo.example.com"] to perform the TLS handshake.
  8. Since the client and "foo.example.com" TLS config match, the hanshake is successful and the controller continues to process the client's connection request.
  9. The controller performs a list/watch for all VirtualHost objects across all gateway.spec.allowedNamespaces and stores the results in a cache.
  10. The controller checks the cache for a VirtualHost with a hostname of "foo.example.com" and finds a hit, a VitualHost named "foo".
  11. The controller uses the VitualHost rules to perform manipulation of the request (optional) and forwards the request to an in-cluster object, i.e. Service.
  12. The in-cluster object receives the request.

TBD's:

  • If the Gateway must terminate the client's TLS connection and establish a new TLS connection to the in-cluster object, i.e. "reencrypt", should the reencrypt config be a property of Gateway or VirtualHost? We must first answer the question "Who is responsible for managing gateway<>in-cluster object security?".
  • Should gateway.spec.hostedNames be more specific, i.e. domainNames, hostNames, etc.?

Major Changes to Existing Service APIs

  • Gateway is no longer responsible for managing xRoute, i.e. VirtualHost association. A Gateway host's names that clients connect to. These names have optional TLS configs. A Gateway uses a hosted name to match VirtualHosts that exist within allowed namespaces.
  • VirtualHost replaces xRoute. A VirtualHost is now responsible for "requesting" a Gateway to be serviced by, i.e. binding. The Dev persona role will contain an RBAC rule allowing users to view Gateway objects and choose a Gateway for the VirtualHost. If the Gateway does not contain a hosted name that matches the hostname of the VirtualHost, status is reflected accordingly.

Notes

Use Cases:

  1. Openshift Online *.example.com

TLS config have no ext refs

Think about how to support GRPC:

  • ref Istio

http3?

hostedNames: # Specifies what names are hosted by this Gateway. # For http connection requests, a hosted name maps to the host header. # For tcp/tls sonnection requests, a hosted name maps to the SNI server_name: # See rfc 6066 for additional details: https://tools.ietf.org/html/rfc6066 # Use "" as a wildcard specifier. For example, "" will host any domain and ".example.com" # will host all subdomains for the "example.com" domain. # Use "." preceding the hosted domain to allow all hostnames of the specified domain. For exammple, # ".example.com" will host any hostname in domain "example.com", i.e. "foo.example.com". # TLS config can be associated to each hosted name (optional).

  tls:
    # https://github.com/kubernetes-sigs/service-apis/issues/51
    insecureConnectionPolicy: redirect
    # https://github.com/kubernetes-sigs/service-apis/issues/52
    terminationPolicy: passthrough
    certificates:
      - group: core
        resource: Secret
        name: foo-cert
    minimumVersion: TLS1_3

allowedNamespaces: Specifies the namespaces that are allowed to bind virtual hosts and TLS configs to this gateway. If unset, defaults to allowing virtual hosts from the same namespace as Gateway. Use "*" to allow binding of virtual hosts from all namespaces.

A listener exposes a Gateway on a specified protocol, port, address, etc.. Think of a listener as a network endpoint or socket.
listeners:

Raw
01_example.yaml
kind: GatewayClass
apiVersion: networking.x.k8s.io/v1alpha1
metadata:
name: acme-tls-lb
controller: acme.io/gateway-controller
parameters:
apiGroup: networking.acme.io
kind: AcmeLBConfig
name: acme-tls-lb-config
---
kind: Gateway
apiVersion: networking.x.k8s.io/v1alpha1
metadata:
name: my-tls-gateway
namespace: my-gw-ns
spec:
class: acme-tls-lb
hostedNames:
- name: foo.example.com
listeners:
- my-tls-listener
tlsConfig:
name: default
namespace: default
- name: bar.example.com
tlsConfig:
name: my-tls
namespace: my-ns
allowedNamespaces:
- foo
- bar
listeners:
- name: my-tls-listener
protocol: TCP
port: 443
- name: my-http-listener
protocol: TCP
port: 80
---
# Alternatively, TLSConfig could be a distinct API object.
# Typically, the cluster op will define a default TLS config.
kind: TLSConfig
apiVersion: networking.x.k8s.io/v1alpha1
metadata:
name: default
namespace: default
spec:
names:
- *.apps.example.com
insecureConnectionPolicy: Redirect
certificates: |-
------ BEGIN CERTIFICATE -----
...
------ END CERTIFICATE -----
minimumVersion: TLS1_2
---
# Based on policy, developers may be permitted to create
# custom TLS configs:
kind: TLSConfig
apiVersion: networking.x.k8s.io/v1alpha1
metadata:
name: my-tls
namespace: my-ns
spec:
names:
- myapp.com
insecureConnectionPolicy: Prohibit
certificates: |-
------ BEGIN CERTIFICATE -----
...
------ END CERTIFICATE -----
minimumVersion: TLS1_2
---
apiVersion: networking.x.k8s.io/v1alpha1
kind: VirtualHost
metadata:
name: foo
namespace: foo
spec:
# Hostnames must match the associated Gateway hostedNames to establish a Gateway<>VirtualHost binding.
hostnames:
- foo.example.com
gateways:
- namespace: my-gw-ns
name: my-tls-gateway
rules:
- match:
path: /foo
filter:
headers:
add:
my-added-header: foo
remove:
- bar
action:
forwardTo:
resource: service
group: core
name: foo
---
apiVersion: networking.x.k8s.io/v1alpha1
kind: VirtualHost
metadata:
name: bar
namespace: bar
spec:
hostnames:
# Hostnames must match the associated Gateway hostedNames to establish a Gateway<>VirtualHost binding.
- bar.example.com
gateways:
- my-gw-ns/my-tls-gateway
rules:
- match:
path: /bar
filter:
headers:
add:
my-added-header: bar
remove:
- foo
action:
forwardTo:
resource: service
group: core
name: bar
Raw
02_example.yaml
kind: GatewayClass
apiVersion: networking.x.k8s.io/v1alpha1
metadata:
name: acme-tls-lb
controller: acme.io/gateway-controller
parameters:
apiGroup: networking.acme.io
kind: AcmeLBConfig
name: acme-tls-lb-config
---
kind: AcmeLBConfig
apiVersion: networking.acme.io/v1alpha1
metadata:
name: acme-tls-lb-config
namespace: my-gw-ns
spec:
allowedGatewayNamespaces:
- my-gw-ns
allowedVitualHostNamespaces:
- foo
- bar
---
kind: Gateway
apiVersion: networking.x.k8s.io/v1alpha1
metadata:
name: my-tls-gateway
namespace: my-gw-ns
spec:
class: acme-tls-lb
hostedNames:
- name: foo.example.com
listeners:
- my-tls-listener
- name: bar.example.com
listeners:
- name: my-tls-listener
protocol: TCP
port: 443
- name: my-http-listener
protocol: TCP
port: 80
status:
hostedNames:
- name: foo.example.com
virtualHost:
---
# Alternatively, TLSConfig could be a distinct API object.
# Typically, the cluster op will define a default TLS config.
kind: TLSConfig
apiVersion: networking.x.k8s.io/v1alpha1
metadata:
name: default
namespace: default
spec:
names:
- *.apps.example.com
insecureConnectionPolicy: Redirect
certificates: |-
------ BEGIN CERTIFICATE -----
...
------ END CERTIFICATE -----
minimumVersion: TLS1_2
---
# Based on policy, developers may be permitted to create
# custom TLS configs:
kind: TLSConfig
apiVersion: networking.x.k8s.io/v1alpha1
metadata:
name: my-tls
namespace: my-ns
spec:
names:
- myapp.com
insecureConnectionPolicy: Prohibit
certificates: |-
------ BEGIN CERTIFICATE -----
...
------ END CERTIFICATE -----
minimumVersion: TLS1_2
---
apiVersion: networking.x.k8s.io/v1alpha1
kind: VirtualHost
metadata:
name: foo
namespace: foo
spec:
# Hostnames must match the associated Gateway hostedNames to establish a Gateway<>VirtualHost binding.
hostnames:
- foo.example.com
gateways:
- namespace: my-gw-ns
name: my-tls-gateway
rules:
- match:
path: /foo
filter:
headers:
add:
my-added-header: foo
remove:
- bar
action:
forwardTo:
resource: service
group: core
name: foo
---
apiVersion: networking.x.k8s.io/v1alpha1
kind: VirtualHost
metadata:
name: bar
namespace: bar
spec:
hostnames:
# Hostnames must match the associated Gateway hostedNames to establish a Gateway<>VirtualHost binding.
- bar.example.com
gateways:
- my-gw-ns/my-tls-gateway
rules:
- match:
path: /bar
filter:
headers:
add:
my-added-header: bar
remove:
- foo
action:
forwardTo:
resource: service
group: core
name: bar
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment