Kubernetes: Pre-pull images into node (moved to: https://github.com/itaysk/kube-imagepuller )

prepull.yaml

###
# There's a newer version available here:
# https://github.com/itaysk/kube-imagepuller
# All future updates will be made there.
# Please also post you questions as issues on that repo instead of commenting here
###


apiVersion: apps/v1beta2
kind: DaemonSet
metadata:
  name: prepull
  annotations:
    source: "https://gist.github.com/itaysk/7bc3e56d69c4d72a549286d98fd557dd"
spec:
  selector:
    matchLabels:
      name: prepull 
  template:
    metadata:
      labels:
        name: prepull 
    spec:
      initContainers:
      - name: prepull 
        image: docker
        command: ["docker", "pull", "hello-world"]
        volumeMounts:
        - name: docker
          mountPath: /var/run
      volumes:
      - name: docker
        hostPath:
          path: /var/run
      containers:
      - name: pause
        image: gcr.io/google_containers/pause

I've an error, how to resolve it?

Events:
  Type     Reason                 Age              From                                         Message
  ----     ------                 ----             ----                                         -------
  Normal   SuccessfulMountVolume  27s              kubelet, bjhtyd-hope-16-229.hadoop.jd.local  MountVolume.SetUp succeeded for volume "docker"
  Normal   SuccessfulMountVolume  27s              kubelet, bjhtyd-hope-16-229.hadoop.jd.local  MountVolume.SetUp succeeded for volume "default-token-4hxt6"
  Normal   Pulling                25s              kubelet, bjhtyd-hope-16-229.hadoop.jd.local  pulling image "docker"
  Warning  Failed                 10s              kubelet, bjhtyd-hope-16-229.hadoop.jd.local  Failed to pull image "docker": rpc error: code = Unknown desc = Error response from daemon: Get https://registry-1.docker.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
  Warning  Failed                 10s              kubelet, bjhtyd-hope-16-229.hadoop.jd.local  Error: ErrImagePull
  Normal   SandboxChanged         9s               kubelet, bjhtyd-hope-16-229.hadoop.jd.local  Pod sandbox changed, it will be killed and re-created.
  Normal   BackOff                6s (x3 over 8s)  kubelet, bjhtyd-hope-16-229.hadoop.jd.local  Back-off pulling image "docker"
  Warning  Failed                 6s (x3 over 8s)  kubelet, bjhtyd-hope-16-229.hadoop.jd.local  Error: ImagePullBackOff

Kubernetes Version is Server Version: version.Info{Major:"1", Minor:"9"

and I tried docker pull docker

Using default tag: latest
Error response from daemon: Get https://registry-1.docker.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)

What should I do if the image being pulled is in a private registry requiring an ImagePullSecret?

@itaysk thanks for this! I'm giving a go in the next few days by wrapping this into an Helm chart, hopefully I'll get to a point where one easily change the list of docker images, etc...I may share that if it turns out to be working/useful.

I have a silly question: I see in the example in the k8s documentation for DaemonSet that it's using apps/v1 instead of apps/v1beta2 in this gist, any reason to not use this?

The best way I've ever seen! Thanks. 💯

@xoen I guess daemonset version was apps/v1beta2 when this yaml was made.

What should I do if the image being pulled is in a private registry requiring an ImagePullSecret?

take a look at this:
https://gist.github.com/pengsun/6438c8571a730f4d4c2d22482d36146b

Hi all, I'm sorry I didn't see your comments, there was something wrong with my GitHub notification settings.
I will address your questions as well as provide an updated version for this gist in a new repo: https://github.com/itaysk/kube-imagepuller.

@quasiben : this is by default, no need to specify this.

@qye1103 : I'm not sure I understand your question

@wugx : seems like there's something wrong with your access to Docker Hub. Are you able to pull other public images from Docker Hub?

@adamnarkunskii : That's a good question, check out the updated version at https://github.com/itaysk/kube-imagepuller

@xoen : Daemonset transitioned between apps, extentions, beta and stable along the different version of Kubernetes. Best to check the correct version for you by checking the online docs for your version.

Also, I've created a Helm chart for this here: https://github.com/itaysk/kube-imagepuller which detects the correct version

@pengsun : not sure that's the best solution to use a ReplicaSet here, I prefer DaemonSet. Second, I think using podspec.imagepullsecret won't work here because the docker init container is not using this secret. I've posted an update to address this here https://github.com/itaysk/kube-imagepuller

I just tried this on GCP but I am getting:

Error response from daemon: Head https://us-central1-docker.pkg.dev/v2/PROJECT_ID/...0.0.1: denied: Permission "artifactregistry.repositories.downloadArtifacts" denied on resource "projects/PROJECT_ID/locations/us-central1/repositories/REPOSITORY" (or it may not exist)

The image does exist definitely, it's running on other nodes, so it appears that the DaemonSet simply does not have the permission to run docker pull. Any idea who I can make this work?

@itaysk also I was wondering if we could do something like this:

initContainers:
- name: prepull
  image:  "<image-url>"
  env:
    SHUTDOWN: "True"  # Signal your application to shutdown immediately

Basically, use the image that you would like to pull directly but set an environment variable to make sure your application shuts-down (or does never actually start). This way the image should get pulled too I guess?

Update: Okay, this appears to work but I'd prefer the other way. Using env is just a bit ugly.