This was initially posted in an kubernetes-sigs/kustomize issue.
We are using Kustomize's vars
feature. Initially we didn't understand how to use it for our purpose, but it is a 100% fit. One example is our Ingress resource, which looks like this:
# file: base/ingress.yaml
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: services
annotations:
kubernetes.io/ingress.global-static-ip-name: $(SERVICES_GLOBAL_STATIC_IP_NAME)
kubernetes.io/ingress.allow-http: "false"
ingress.gcp.kubernetes.io/pre-shared-cert: $(SERVICES_PRE_SHARED_CERT)
kubernetes.io/ingress.class: "gce"
spec:
rules:
- host: $(HOST_A)
http:
paths:
- backend:
serviceName: serviceA
servicePort: 80
- host: $(HOST_B)
http:
paths:
- backend:
serviceName: serviceB
servicePort: 80
- host: $(HOST_C)
http:
paths:
- backend:
serviceName: serviceC
servicePort: 80
Then our configMapGenerator / vars looks like this:
# file: base/kustomization.yaml
bases:
- ingress.yaml
configMapGenerator:
- name: ops-ingress-properties
envs: [environment.properties]
vars:
- name: SERVICES_GLOBAL_STATIC_IP_NAME
objref: { kind: ConfigMap, name: ops-ingress-properties, apiVersion: v1 }
fieldref: { fieldpath: data.SERVICES_GLOBAL_STATIC_IP_NAME }
- name: SERVICES_PRE_SHARED_CERT
objref: { kind: ConfigMap, name: ops-ingress-properties, apiVersion: v1 }
fieldref: { fieldpath: data.SERVICES_PRE_SHARED_CERT }
- name: HOST_A
objref: { kind: ConfigMap, name: ops-ingress-properties, apiVersion: v1 }
fieldref: { fieldpath: data.HOST_A }
- name: HOST_B
objref: { kind: ConfigMap, name: ops-ingress-properties, apiVersion: v1 }
fieldref: { fieldpath: data.HOST_B }
- name: HOST_C
objref: { kind: ConfigMap, name: ops-ingress-properties, apiVersion: v1 }
fieldref: { fieldpath: data.HOST_C }
and the properties like this:
# file: base/environment.properties
# Ingress annotations
SERVICES_GLOBAL_STATIC_IP_NAME=services
SERVICES_PRE_SHARED_CERT=a-yyyymmdd,b-yyyymmdd,c-yyyymmdd
# Hosts
HOST_A=a.example.org
HOST_B=b.example.org
HOST_C=c.example.org
then in our overlays we redefine the environment.properties
file and have this in Kustomization:
# file: overlay/staging/kustomization.yaml
configMapGenerator:
- name: ops-ingress-properties
envs: [environment.properties]
behavior: replace # <======= critical
which overwrites the values in the base like this:
# file: overlay/staging/environment.properties
# Ingress annotations
SERVICES_GLOBAL_STATIC_IP_NAME=services-staging
SERVICES_PRE_SHARED_CERT=a-staging-yyyymmdd,b-staging-yyyymmdd,c-staging-yyyymmdd
# Hosts
HOST_A=a-staging.example.org
HOST_B=b-staging.example.org
HOST_C=c-staging.example.org
This works ideal for us! A bit sad that it took us soo long to discover this feature. We really don't want it replaced/removed.
@jowbowbeer -- thanks for the eschewedfeatures link, which explains the reasoning. Makes sense. The article says that one of the bad things about unstructured parameterization and I quote "The source yaml gets polluted with $VARs and can no longed be applied as is to the cluster (it must be processed)." On the surface, it looks like the technique is doing that -- of course the base/kustomization.yaml explicity references the configmap with an objectref and fieldref to populate the values from. Overall, the technique presented here is still conceptually fairly difficult to grasp -- and I don't like the fact that I end up with a configmap that I don't need -- its just an artifact used in the env var replacements.