Skip to content

Instantly share code, notes, and snippets.

@atheiman
Last active February 11, 2024 01:28
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 4 You must be signed in to fork a gist
  • Save atheiman/a327a6cb5027376192ea396e14830157 to your computer and use it in GitHub Desktop.
Save atheiman/a327a6cb5027376192ea396e14830157 to your computer and use it in GitHub Desktop.
Generate job artifacts in an initContainer and export the files to workstation afterwards.
# Allows copying of job files to local after execution. Example copy command:
#
# kubectl apply -f ./export-file-job.yaml
# POD=$(kubectl get pod --selector=job-name=export-files -o jsonpath='{.items[0].metadata.name}')
# kubectl wait pod/$POD --for=condition=ready
# until kubectl logs $POD | grep 'Ready for download'; do sleep 2; done
# kubectl cp $POD:files.zip ./job-files-$(date +"%Y%m%d%H%M").zip
#
# Another option would be for the `export-files` container to be a webserver to
# serve of the artifact files to be downloaded with `kubectl port-forward ...`.
apiVersion: batch/v1
kind: Job
metadata:
name: export-files
spec:
template:
metadata:
labels:
job-name: export-files
spec:
restartPolicy: Never
volumes:
- name: job-files
emptyDir: {}
initContainers:
- name: job
image: alpine
volumeMounts:
- name: job-files
mountPath: /job
workingDir: /job
command: [/bin/sh, -c]
args:
- |
echo "This is where your job would happen. The log from this job is"
echo "is visible with 'kubectl logs POD job'."
echo "Imagine the files below are generated from the job."
echo contents of a file I want from a job container > ./file-1.txt
date > ./file-2.txt
hostname > ./file-3.txt
echo "Zipping job files"
apk add --quiet --no-cache zip
zip -v ./files.zip $(find . -type f)
ls -alh ./files.zip
echo "Now this initContainer will exit and the container below will start"
echo "and sit idle so that these job files can be cp'd out of the pod."
containers:
- name: export-files
image: alpine
volumeMounts:
- name: job-files
mountPath: /job
workingDir: /job
command: [/bin/sh, -c]
args:
- |
echo "Ready for download"
echo "Sleeping to allow files to be exported with 'kubectl cp'"
sleep 1m
echo "Exiting"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment