Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Using a Local Registry with Minikube

Using a Local Registry with Minikube

Install a local Registry

These instructions include running a local registry accessible from Kubernetes as well as from the host development machine at registry.dev.svc.cluster.local:5000.

  1. Use the docker CLI to run the registry:2 container from Docker, listening on port 5000, and persisting images in the ~/.registry/storage directory.

    docker run -d -p 5000:5000 --restart=always --volume ~/.registry/storage:/var/lib/registry registry:2
    
  2. Edit the /etc/hosts file on your development machine, adding the name registry.dev.svc.cluster.local on the same line as the entry for localhost.

  3. Validate that the registry is running.

    docker ps
    
    CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                    NAMES
    02ea46d51f58        registry:2          "/entrypoint.sh /etc…"   About an hour ago   Up About a minute   0.0.0.0:5000->5000/tcp   sharp_pike
    
  4. Validate that the registry at registry.dev.svc.cluster.local:5000 is reachable from your development machine.

    curl registry.dev.svc.cluster.local:5000/v2/_catalog
    
    {"repositories":[]}
    
  5. Configure the docker daemon with an insecure registy at registry.dev.svc.cluster.local:5000.

    On macOS in ~/.docker/daemon.json on Linux in /etc/docker/daemon.json (craete the file if it does not exist)

    {
      "insecure-registries": ["registry.dev.svc.cluster.local:5000"]
    }
    

Start Minikube

minikube start --cpus 4 --memory 4096 --insecure-registry registry.dev.svc.cluster.local:5000

Configure a fixed IP address

This IP address will allow processes in Minikube to reach the registry running on your host. Configuring a fixed IP address avoids the problem of the IP address changing whenever you connect your machine to a different network. If your machine already uses the 172.16.x.x range for other purposes, choose an address in a different range e.g. 172.31.x.x..

export DEV_IP=172.16.1.1

Create an alias on MacOS:

sudo ifconfig lo0 alias $DEV_IP

Create an alias on Linux:

sudo ifconfig lo:0 $DEV_IP

Note that the alias will need to be reestablished when you restart your machine. This can be avoided by using a launchdeamon on MacOS or by editing /etc/network/interfaces on Linux.

Minikube /etc/hosts

Add an entry to /etc/hosts inside the minikube VM, pointing the registry to the IP address of the host. This will result in registry.dev.svc.cluster.local resolving to the host machine allowing the docker daemon in minikube to pull images from the local registry. This uses the DEV_IP environment variable from the previous step.

export DEV_IP=172.16.1.1
minikube ssh "echo \"$DEV_IP       registry.dev.svc.cluster.local\" | sudo tee -a  /etc/hosts"

Kubernetes Service and Endpoint

Create a kubernetes service without selectors called registry in the dev namespace and a kubernetes endpoint with the same name pointing to the static IP address of your development machine. This will result in registry.dev.svc.cluster.local resolving to the host machine, allowing container builds running in the cluster, to work with the local registry.

kubectl create namespace dev
cat <<EOF | kubectl apply -n dev -f -
---
kind: Service
apiVersion: v1
metadata:
  name: registry
spec:
  ports:
  - protocol: TCP
    port: 5000
    targetPort: 5000
---
kind: Endpoints
apiVersion: v1
metadata:
  name: registry
subsets:
  - addresses:
      - ip: $DEV_IP
    ports:
      - port: 5000
EOF

Relocate app images

Install irel CLI from https://github.com/pivotal/image-relocation/releases

For this example we are using a registry prefix of registry.dev.svc.cluster.local:5000; you would need to change this to match your registry (you also need to be authenticated with this registry).

To copy the time sink app to your own registry run:

irel copy springcloudstream/time-source-rabbit:2.1.2.RELEASE registry.dev.svc.cluster.local:5000/time-source-rabbit:2.1.2.RELEASE

To copy the log sink app to your own registry run:

irel copy springcloudstream/log-sink-rabbit:2.1.3.RELEASE registry.dev.svc.cluster.local:5000/log-sink-rabbit:2.1.3.RELEASE

You can now register the relocated apps using SCDF Shell:

dataflow:>app register --name time --type source --uri docker:registry.dev.svc.cluster.local:5000/time-source-rabbit:2.1.2.RELEASE
dataflow:>app register --name log --type sink --uri docker:registry.dev.svc.cluster.local:5000/log-sink-rabbit:2.1.3.RELEASE
@Chasun-fhd
Copy link

Chasun-fhd commented May 3, 2022

@Chasun-fhd verify that your image is in the registry by curl http://localhost:5001/v2/_catalog || curl http://127.0.0.1:5001/v2/_catalog

i tried too, returns the same like http://registry.dev.svc.cluster.local:5001/v2/_catalog

@Chasun-fhd
Copy link

Chasun-fhd commented May 3, 2022

finally, i solved this problem. Configure static ip address on my mac. then replace ip in these instructions with new static ip. it worked.

@oscarCCOnis
Copy link

oscarCCOnis commented Oct 11, 2022

I have my image in my repo, but my minikube is trying to find https instead of http and its failing

$ curl registry.dev.svc.cluster.local:5000/v2/_catalog
{"repositories":["hpesaconnmonitor"]}

Events:                                                                                                                                  │
│   Type     Reason     Age              From               Message                                                                        │
│   ----     ------     ----             ----               -------                                                                        │
│   Normal   Scheduled  5s               default-scheduler  Successfully assigned monitoring/monitoring-deployment-696c48f8d4-flp5v to min │
│ ikube                                                                                                                                    │
│   Normal   Pulling    5s               kubelet            Pulling image "registry.dev.svc.cluster.local:5000/hpesaconnmonitor:1.0"       │
│   Warning  Failed     4s               kubelet            Failed to pull image "registry.dev.svc.cluster.local:5000/hpesaconnmonitor:1.0 │
│ ": rpc error: code = Unknown desc = Error response from daemon: Get "https://registry.dev.svc.cluster.local:5000/v2/": dial tcp: lookup  │
│ registry.dev.svc.cluster.local: no such host                                                                                             │
│   Warning  Failed     4s               kubelet            Error: ErrImagePull                                                            │
│   Normal   BackOff    3s (x2 over 4s)  kubelet            Back-off pulling image "registry.dev.svc.cluster.local:5000/hpesaconnmonitor:1 │
│ .0"                                                                                                                                      │
│   Warning  Failed     3s (x2 over 4s)  kubelet            Error: ImagePullBackOff`

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment