Configuring the LoadBalancer created by the Consul API Gateway controller

README.md

Configuring the load balancer created by the Consul API Gateway controller

Unless configured otherwise, the Consul API Gateway controller creates a load balancer for each Gateway resource. Since we're in Kubernetes, we do this by spinning up a Service resource with type: LoadBalancer.

Each of the major cloud providers allow you to configure the resulting load balancer by specifying one or more annotations on the Kubernetes Service object. You may not add these annotations directly to the Service as the controller is creating it on your behalf; however, the Consul API Gateway controller will copy specific annotations from the Gateway object - which you do control - to the Service object that it creates. To do so, you must specify the list of keys for annotations that you would like copied in values.yaml here. You must then add those annotations to the Gateway. The API gateway controller will then copy any annotations from the Gateway to the Service where the key of the annotation is listed in values.yaml.

In this example, we will configure our load balancer in Google Kubernetes Engine to be of type "internal" (docs).

$ helm upgrade --install --values ./values.yaml consul hashicorp/consul --version "0.49.0" --namespace consul --create-namespace

We can now verify that the GatewayClassConfig which we'll use for our Gateway contains the allow-list:

$ kubectl get gatewayclassconfig consul-api-gateway -o yaml
apiVersion: api-gateway.consul.hashicorp.com/v1alpha1
kind: GatewayClassConfig
...
spec:
  ...
  copyAnnotations:
    service:
    - networking.gke.io/load-balancer-type
  serviceType: LoadBalancer

Now we can create a Gateway and see the annotations that were copied from our Gateway resource to the Service created by the controller:

$ kubectl apply --filename ./gateway.yaml

$ kubectl get service internal-gateway -o yaml
apiVersion: v1
kind: Service
metadata:
  name: internal-gateway
  namespace: consul
  annotations:
    networking.gke.io/load-balancer-type: Internal
  ...
spec:
  type: LoadBalancer
  ...

gateway.yaml

apiVersion: gateway.networking.k8s.io/v1beta1
kind: Gateway
metadata:
  name: internal-gateway
  namespace: consul
  annotations:
    # This annotation's key *is* listed in values.yaml, so it will be copied to the service
    networking.gke.io/load-balancer-type: "Internal"
    # This annotation's key is *not* listed in values.yaml, so it will not be copied to the Service
    uncopied-annotation: "this does not get copied"
spec:
  ...

values.yaml

...

apiGateway:
  enabled: true
  image: "hashicorp/consul-api-gateway:0.4"
  managedGatewayClass:
    copyAnnotations:
      # Note the structure difference here compared to the inaccurate
      # example fixed by https://github.com/hashicorp/consul-k8s/pull/1631
      service:
        # This list only includes the *keys* of the annotations to be copied.
        # The values will be pulled from the Gateway's annotations.
        annotations: |
          - networking.gke.io/load-balancer-type