This was initially posted in an kubernetes-sigs/kustomize issue.
We are using Kustomize's vars
feature. Initially we didn't understand how to use it for our purpose, but it is a 100% fit. One example is our Ingress resource, which looks like this:
# file: base/ingress.yaml
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: services
annotations:
kubernetes.io/ingress.global-static-ip-name: $(SERVICES_GLOBAL_STATIC_IP_NAME)
kubernetes.io/ingress.allow-http: "false"
ingress.gcp.kubernetes.io/pre-shared-cert: $(SERVICES_PRE_SHARED_CERT)
kubernetes.io/ingress.class: "gce"
spec:
rules:
- host: $(HOST_A)
http:
paths:
- backend:
serviceName: serviceA
servicePort: 80
- host: $(HOST_B)
http:
paths:
- backend:
serviceName: serviceB
servicePort: 80
- host: $(HOST_C)
http:
paths:
- backend:
serviceName: serviceC
servicePort: 80
Then our configMapGenerator / vars looks like this:
# file: base/kustomization.yaml
bases:
- ingress.yaml
configMapGenerator:
- name: ops-ingress-properties
envs: [environment.properties]
vars:
- name: SERVICES_GLOBAL_STATIC_IP_NAME
objref: { kind: ConfigMap, name: ops-ingress-properties, apiVersion: v1 }
fieldref: { fieldpath: data.SERVICES_GLOBAL_STATIC_IP_NAME }
- name: SERVICES_PRE_SHARED_CERT
objref: { kind: ConfigMap, name: ops-ingress-properties, apiVersion: v1 }
fieldref: { fieldpath: data.SERVICES_PRE_SHARED_CERT }
- name: HOST_A
objref: { kind: ConfigMap, name: ops-ingress-properties, apiVersion: v1 }
fieldref: { fieldpath: data.HOST_A }
- name: HOST_B
objref: { kind: ConfigMap, name: ops-ingress-properties, apiVersion: v1 }
fieldref: { fieldpath: data.HOST_B }
- name: HOST_C
objref: { kind: ConfigMap, name: ops-ingress-properties, apiVersion: v1 }
fieldref: { fieldpath: data.HOST_C }
and the properties like this:
# file: base/environment.properties
# Ingress annotations
SERVICES_GLOBAL_STATIC_IP_NAME=services
SERVICES_PRE_SHARED_CERT=a-yyyymmdd,b-yyyymmdd,c-yyyymmdd
# Hosts
HOST_A=a.example.org
HOST_B=b.example.org
HOST_C=c.example.org
then in our overlays we redefine the environment.properties
file and have this in Kustomization:
# file: overlay/staging/kustomization.yaml
configMapGenerator:
- name: ops-ingress-properties
envs: [environment.properties]
behavior: replace # <======= critical
which overwrites the values in the base like this:
# file: overlay/staging/environment.properties
# Ingress annotations
SERVICES_GLOBAL_STATIC_IP_NAME=services-staging
SERVICES_PRE_SHARED_CERT=a-staging-yyyymmdd,b-staging-yyyymmdd,c-staging-yyyymmdd
# Hosts
HOST_A=a-staging.example.org
HOST_B=b-staging.example.org
HOST_C=c-staging.example.org
This works ideal for us! A bit sad that it took us soo long to discover this feature. We really don't want it replaced/removed.
@joebowbeer I just looked at the latest source -- if you leave the values out that it will take the env vars, but give a big fat warning that this will not be supported in future kustomize versions.
@hermanbanken - thanks for providing this solution.
I have a question about all of this. I just tried the technique and it worked just as shown. However, I only need variable replacements and do not require the generated configmap. This shown method also has a lot of moving parts. You must have all of the pieces in the correct positions - it took me a few tries to get this right! Is there no simple mechanism in kustomize that allows variable replacement? One could keep
$(ENV VAR)
syntax be used in the base folder yaml, and one would have an environment.properties file in the overlay with ENV=value pairs (exactly as in this sample). The difference would be that kustomize.yaml in the overlay would reference environment.properties without a configmap generator. The base kustomize.yaml would also not have the config map generator. The desired effect is to use the environment variables in the base manifests and have the all instances replaced in all manifests in the base with the values from environment.properties. Essentially this would just be a glorified search/replace of variables.Maybe this functionality exists -- if so, please enlighten me.