Mounting a Secret into a Directory that Already Contains Files on an Image You Don't Control

mounting-a-file.md

Mounting a Secret into a Directory that Already Contains Files on an Image You Don't Control

In some situations you may want to mount a value from a Secret or ConfigMap as a file in a directory which contains files you'd rather not overwrite. For this, you can use a subPath when defining the volume mount. Lets assume we have a secret my-secret which contains a key ca_bundle.pem.

volumes:
- name: my-secret-volume
  secret:
    secretName: my-secret

volumeMounts:
- name: my-secret-volume
  mountPath: /etc/ssl/certs/ca_bundle.pem
  subPath: ca_bundle.pem

The above volume mount will extract the key named in subpath from the secret, and mount it at mountPath without destroying the rest of the contents of that directory. In this case, ca_bundle.pem from my-secret gets mounted at /etc/ssl/certs/ca_bundle.pem.

Normally, when a Secret or ConfigMap is mounted as a volume, the files inside will be updated whenever the Secret or ConfigMap is updated. However, files mounted as subpaths will not receive updates due to some of the specifics of how that mounting works. It would be nice to be able to mount the Secret as a volume normally, and then create a symlink to that file so that we receive updates. One approach would be to adjust the image itself to put a symlink where we want, but that's only possible if we control the image. Instead, we can use an Init Container to create the symlink for us when the Pod starts up.

containers:
  - image: my-container-image:1.1
    <snip>
    volumeMounts:
      - name: certs
        mountPath: /etc/ssl/certs
      - name: credentials
        mountPath: /credentials
initContainers:
  - image: my-container-image:1.1 (important to use the same image rather than a busybox or etc)
    name: setup-secret-symlink
    command:
      - sh
      - '-ec'
      - >-
        cp /etc/ssl/certs/* /certs/;
        ln -sf /credentials/ca_bundle.pem /certs/ca_bundle.pem;
    volumeMounts:
      - name: certs
        mountPath: /certs
      - name: credentials
        mountPath: /credentials
volumes:
  - name: certs
    emptyDir: {}
  - name: credentials
    secret:
      secretName: credentials

It would make sense to create an emptyDir mount, use the init container to write a symlink to our secret volume in the emptyDir, and then mount that symlink as a subpath where we want it on the container. However, for security reasons Kubernetes won't let us mount a symlink to a subpath-- we would still have to clobber the files that were originally in /etc/ssl/certs, which we are trying to avoid. Instead, we can copy the contents of /etc/ssl/certs on the init container to the emptyDir, write the symlink where we want it, and then mount that entire volume over top of /etc/ssl/certs on the main container. This effectively replaces the version of /etc/ssl/certs with the one from the init container, plus our symlink.

(Note: The security issue being worked around is specific to the way symlinks are resolved as part of mounting subpaths and doesn't impact normal mounts, so we're not sidestepping a security measure with this work around.)