Skip to content

Instantly share code, notes, and snippets.

@janosroden
Last active March 11, 2019 17:47
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save janosroden/78725e3f846763aa3a660a6b2116c7da to your computer and use it in GitHub Desktop.
Save janosroden/78725e3f846763aa3a660a6b2116c7da to your computer and use it in GitHub Desktop.
Example solution (not copy-paste ready) to stop sidecar containers in a job (https://github.com/kubernetes/kubernetes/issues/25908)
apiVersion: batch/v1beta1
kind: CronJob
spec:
concurrencyPolicy: Forbid
jobTemplate:
spec:
template:
spec:
containers:
- name: &jobContainer job
- name: &sidecarContainer postfix
- name: docker
image: docker
securityContext:
privileged: true
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
- name: JOB_CONTAINER
value: *jobContainer
- name: SIDECAR_CONTAINER
value: *sidecarContainer
command:
- sh
- -c
- |
set -e
set -o pipefail
# getContainerProp <containerName> <propertyName>
# propertyName: one of https://docs.docker.com/engine/reference/commandline/ps/#formatting
function getContainerProp() {
docker ps --all --filter "label=io.kubernetes.pod.namespace=$POD_NAMESPACE" --filter "label=io.kubernetes.pod.name=$POD_NAME" --filter "label=io.kubernetes.container.name=$1" --format "{{$2}}"
}
# waitForStatus <containerName> <statusRegex>
# statusRegex: see statuses with docker ps -a
function waitForStatus() {
while ! getContainerProp $1 .Status | grep -q "$2"; do
echo "Waiting for status of $1 get match to $2..."
sleep 2
done
echo "Wait for status of $1 is done."
}
# waitForExited <containerName>
function waitForExited() {
waitForStatus $1 "^Exited"
}
# waitForExited <containerName>
function waitForRunning() {
waitForStatus $1 "^Up "
}
# execInContainer <containerName> <command> [args...]
function execInContainer() {
local cId=$(getContainerProp $1 .ID)
shift
docker exec $cId "$@"
}
# stopContainer <containerName>
function stopContainer() {
echo "Stopping $1..."
docker stop -t 60 $(getContainerProp $1 .ID)
}
# Wait for jobs
waitForExited $JOB_CONTAINER
# Wait for services
waitForRunning $SIDECAR_CONTAINER
while ! execInContainer $SIDECAR_CONTAINER postqueue -p > /dev/null 2>&1; do
echo "Waiting for postfix init..."
sleep 1
done
# Wait for services to get done their jobs
while ! execInContainer $SIDECAR_CONTAINER postqueue -p | grep -q 'Mail queue is empty'; do
echo "Waiting for sending mails..."
sleep 2
done
# Stop services
stopContainer $SIDECAR_CONTAINER
volumeMounts:
- name: docker
mountPath: /var/run/docker.sock
volumes:
- name: docker
hostPath:
path: /var/run/docker.sock
type: Socket
@janosroden
Copy link
Author

It uses a privileged container with a mounted docker socket to manage the containers in the job. Note that this is an ash script (busybox)
Limitations:

  • Requires docker
  • Requires no concurrent run or at least not on the same node
  • Requires some scripting

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment