image-copy.md

TLDR oc image mirror between registry works well, but if you try to redirect to intermidate local file, to then be served locally, the digest is not maintained. Serving locally is experimental as written in the help.

Having intermediate file is useful if in your enterprise you cannot proxy to the internet and download public images into a registry.

Workarounds

• It is possible to stand up docker registry that references a volume. Perform the oc image mirror quay privaterepo. Tar up the data volume, move it inside the enterprise, and mount it with the docker registry running inside the enterprise, and perform oc image mirror privaterepo enterpriserepo.
• It should also be possible to create a custom registry that includes the etcd-operator and etcd images preloaded.

1. backup the data folder, after image is loaded

# ls -al
total 63720
drwxr-xr-x. 6 root root       92 Jun 26 20:15 .
drwxr-xr-x. 3 root root       22 Jun 26 20:11 ..
drwxr-xr-x. 2 root root       22 Jun 26 20:13 auth
drwxr-xr-x. 2 root root       46 Jun 26 20:12 certs
drwxr-xr-x. 2 root root       24 Jun 26 20:12 config
drwxr-xr-x. 3 root root       20 Jun 26 20:14 data
-rw-r--r--. 1 root root 65249280 Jun 26 20:15 data.etcd.etcd-operator.tar

2. copy and untar data folder, for new registry.

# ls -al
total 0
drwxr-xr-x. 6 root root 57 Jun 26 16:43 .
drwxr-xr-x. 3 root root 62 Jun 26 16:38 ..
drwxr-xr-x. 2 root root 22 Jun 26 16:09 auth
drwxr-xr-x. 2 root root 46 Jun 26 16:40 certs
drwxr-xr-x. 2 root root 24 Jun 26 16:27 config
drwxr-xr-x. 3 root root 20 Jun 26 16:30 data
# tar xvf ../data.with.etcd.etcd-operator.tar
# podman restart registry
81ef825eef6d5e723ac80d5f670326593ec68722b1662cb2847f183524359851
# podman login -u user -p pass dell-r640-015:5000
Login Succeeded!

3. docker pull from the registry

# podman pull dell-r640-015:5000/coreos/etcd-operator@sha256:66a37fd61a06a43969854ee6d3e21087a98b93838e284a6086b13917f96b0d9b
Trying to pull dell-r640-015:5000/coreos/etcd-operator@sha256:66a37fd61a06a43969854ee6d3e21087a98b93838e284a6086b13917f96b0d9b...
Getting image source signatures
Copying blob e05d0d7eb99a done
Copying blob 80b1b554e5b4 done
Copying blob 59265c40e257 done
Copying blob 678f2cfebea6 done
Copying blob 8ea0315241b4 done
Copying blob a3ed95caeb02 done
Copying blob 6d4b4c91b3ff done
Copying blob a3ed95caeb02 skipped: already exists
Writing manifest to image destination
Storing signatures
9d5c51d92fbddcda022478def5889a9ceb074305d83f2336cfc228827a03d5d5
# podman pull dell-r640-015:5000/coreos/etcd@sha256:e372648462335719b21c5410d5a064a470953a73d59b587611a75f6dfe0d6fba
Trying to pull dell-r640-015:5000/coreos/etcd@sha256:e372648462335719b21c5410d5a064a470953a73d59b587611a75f6dfe0d6fba...
Getting image source signatures
Copying blob 132fdc5dd59a done
Copying blob 50687f8720e4 done
Copying blob 92e421f14f67 done
Copying blob a3ed95caeb02 done
Copying blob df93922902f8 done
Copying blob 2fdfe1cd78c2 done
Copying blob a3ed95caeb02 skipped: already exists
Copying blob a3ed95caeb02 skipped: already exists
Copying blob 91b873c3004b done
Writing manifest to image destination
Storing signatures
30d9f8842f2656c3042197622504db788a659081b5e83ad74e608d6721830c63

Running mirror command

1. copy image from quay to private registry, using oc image mirror QUAY PRIVATEREPO. The oc image mirror supports well identical manifests copy between 1 registry and another registry.

# oc image mirror --force --insecure quay.io/coreos/etcd-operator@sha256:66a37fd61a06a43969854ee6d3e21087a98b93838e284a6086b13917f96b0d9b dell-r640-015:5000/coreos/etcd-operator -a /tmp/auth.json
dell-r640-015:5000/
  coreos/etcd-operator
    blobs:
      quay.io/coreos/etcd-operator sha256:59265c40e257554058624f35856dafd82d135c4ef406de298cb1fee647867381
      quay.io/coreos/etcd-operator sha256:678f2cfebea627b8b3d5bed91a6fe1d3749421fc504a568a7977ce912a953763
      quay.io/coreos/etcd-operator sha256:6d4b4c91b3ff6970e22c704f59d45a5945065cb80bd7055114f6d27432e2205a
      quay.io/coreos/etcd-operator sha256:80b1b554e5b4c4a4334fd25764d6c3d78bafd04ef727714dfb25b1356cc3a265
      quay.io/coreos/etcd-operator sha256:8ea0315241b46a55ff650eddc8d06bfe7bfbdc072b4f3878d6c4598fc7b015d3
      quay.io/coreos/etcd-operator sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4
      quay.io/coreos/etcd-operator sha256:e05d0d7eb99a1d91fa7349dd9a98a8b2d9e222b39fa305543e1ac4c380147a4b
    manifests:
      sha256:66a37fd61a06a43969854ee6d3e21087a98b93838e284a6086b13917f96b0d9b
  stats: shared=0 unique=7 size=0B

phase 0:
  dell-r640-015:5000 coreos/etcd-operator blobs=7 mounts=0 manifests=1 shared=0

info: Planning completed in 1.59s
uploading: dell-r640-015:5000/coreos/etcd-operator sha256:59265c40e257554058624f35856dafd82d135c4ef406de298cb1fee647867381 0B
uploading: dell-r640-015:5000/coreos/etcd-operator sha256:6d4b4c91b3ff6970e22c704f59d45a5945065cb80bd7055114f6d27432e2205a 0B
uploading: dell-r640-015:5000/coreos/etcd-operator sha256:678f2cfebea627b8b3d5bed91a6fe1d3749421fc504a568a7977ce912a953763 0B
uploading: dell-r640-015:5000/coreos/etcd-operator sha256:e05d0d7eb99a1d91fa7349dd9a98a8b2d9e222b39fa305543e1ac4c380147a4b 0B
uploading: dell-r640-015:5000/coreos/etcd-operator sha256:80b1b554e5b4c4a4334fd25764d6c3d78bafd04ef727714dfb25b1356cc3a265 0B
uploading: dell-r640-015:5000/coreos/etcd-operator sha256:8ea0315241b46a55ff650eddc8d06bfe7bfbdc072b4f3878d6c4598fc7b015d3 0B
warning: Layer size mismatch for sha256:6d4b4c91b3ff6970e22c704f59d45a5945065cb80bd7055114f6d27432e2205a: had 0, wrote 1251
uploading: dell-r640-015:5000/coreos/etcd-operator sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 0B
warning: Layer size mismatch for sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4: had 0, wrote 32
warning: Layer size mismatch for sha256:8ea0315241b46a55ff650eddc8d06bfe7bfbdc072b4f3878d6c4598fc7b015d3: had 0, wrote 351520
warning: Layer size mismatch for sha256:59265c40e257554058624f35856dafd82d135c4ef406de298cb1fee647867381: had 0, wrote 2016712
warning: Layer size mismatch for sha256:e05d0d7eb99a1d91fa7349dd9a98a8b2d9e222b39fa305543e1ac4c380147a4b: had 0, wrote 17235178
warning: Layer size mismatch for sha256:678f2cfebea627b8b3d5bed91a6fe1d3749421fc504a568a7977ce912a953763: had 0, wrote 17555180
warning: Layer size mismatch for sha256:80b1b554e5b4c4a4334fd25764d6c3d78bafd04ef727714dfb25b1356cc3a265: had 0, wrote 15521633
sha256:66a37fd61a06a43969854ee6d3e21087a98b93838e284a6086b13917f96b0d9b dell-r640-015:5000/coreos/etcd-operator
info: Mirroring completed in 45.51s (1.158MB/s)

2. verify that we can pull the digest from the private registry

# podman pull dell-r640-015:5000/coreos/etcd-operator@sha256:66a37fd61a06a43969854ee6d3e21087a98b93838e284a6086b13917f96b0d9b
Trying to pull dell-r640-015:5000/coreos/etcd-operator@sha256:66a37fd61a06a43969854ee6d3e21087a98b93838e284a6086b13917f96b0d9b...
Getting image source signatures
Copying blob 80b1b554e5b4 done
Copying blob a3ed95caeb02 done
Copying blob 678f2cfebea6 done
Copying blob 8ea0315241b4 done
Copying blob e05d0d7eb99a done
Copying blob 59265c40e257 done
Copying blob 6d4b4c91b3ff done
Copying blob a3ed95caeb02 skipped: already exists
Writing manifest to image destination
Storing signatures
9d5c51d92fbddcda022478def5889a9ceb074305d83f2336cfc228827a03d5d5

3. Verify the sha in the pulled docker image

[root@dell-r640-015 opt]# podman images
REPOSITORY                                TAG      IMAGE ID       CREATED         SIZE
docker.io/library/registry                2        2d4f4b5309b1   7 days ago      26.8 MB
dell-r640-015:5000/coreos/etcd-operator   <none>   9d5c51d92fbd   16 months ago   140 MB
[root@dell-r640-015 opt]# podman inspect 9d5c51d92fbd
[
    {
        "Id": "9d5c51d92fbddcda022478def5889a9ceb074305d83f2336cfc228827a03d5d5",
        "Digest": "sha256:66a37fd61a06a43969854ee6d3e21087a98b93838e284a6086b13917f96b0d9b",
        "RepoTags": null,
        "RepoDigests": [
            "dell-r640-015:5000/coreos/etcd-operator@sha256:66a37fd61a06a43969854ee6d3e21087a98b93838e284a6086b13917f96b0d9b"
        ],
        "Parent": "",
        "Comment": "",
        "Created": "2019-02-28T02:02:50.022009749Z",

However, when using oc image mirror to redirect to a file, the resulting digest is NOT maintained. The digest should not be different, but it is. Looks like a bug. Redirecting to a file is meant to then be served locally, and that function is experimental.

# oc image mirror --force --insecure quay.io/coreos/etcd-operator@sha256:66a37fd61a06a43969854ee6d3e21087a98b93838e284a6086b13917f96b0d9b file://coreos/etcd-operator -a /tmp/auth.json --dir=verify
<dir>
  coreos/etcd-operator
    blobs:
      quay.io/coreos/etcd-operator sha256:59265c40e257554058624f35856dafd82d135c4ef406de298cb1fee647867381
      quay.io/coreos/etcd-operator sha256:678f2cfebea627b8b3d5bed91a6fe1d3749421fc504a568a7977ce912a953763
      quay.io/coreos/etcd-operator sha256:6d4b4c91b3ff6970e22c704f59d45a5945065cb80bd7055114f6d27432e2205a
      quay.io/coreos/etcd-operator sha256:80b1b554e5b4c4a4334fd25764d6c3d78bafd04ef727714dfb25b1356cc3a265
      quay.io/coreos/etcd-operator sha256:8ea0315241b46a55ff650eddc8d06bfe7bfbdc072b4f3878d6c4598fc7b015d3
      quay.io/coreos/etcd-operator sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4
      quay.io/coreos/etcd-operator sha256:e05d0d7eb99a1d91fa7349dd9a98a8b2d9e222b39fa305543e1ac4c380147a4b
    manifests:
      sha256:66a37fd61a06a43969854ee6d3e21087a98b93838e284a6086b13917f96b0d9b
  stats: shared=0 unique=7 size=0B

phase 0:
   coreos/etcd-operator blobs=7 mounts=0 manifests=1 shared=0

info: Planning completed in 1.62s
uploading: file://coreos/etcd-operator sha256:678f2cfebea627b8b3d5bed91a6fe1d3749421fc504a568a7977ce912a953763 0B
uploading: file://coreos/etcd-operator sha256:6d4b4c91b3ff6970e22c704f59d45a5945065cb80bd7055114f6d27432e2205a 0B
uploading: file://coreos/etcd-operator sha256:59265c40e257554058624f35856dafd82d135c4ef406de298cb1fee647867381 0B
uploading: file://coreos/etcd-operator sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4 0B
uploading: file://coreos/etcd-operator sha256:80b1b554e5b4c4a4334fd25764d6c3d78bafd04ef727714dfb25b1356cc3a265 0B
uploading: file://coreos/etcd-operator sha256:e05d0d7eb99a1d91fa7349dd9a98a8b2d9e222b39fa305543e1ac4c380147a4b 0B
warning: Layer size mismatch for sha256:a3ed95caeb02ffe68cdd9fd84406680ae93d633cb16422d00e8a7c22955b46d4: had 0, wrote 32
uploading: file://coreos/etcd-operator sha256:8ea0315241b46a55ff650eddc8d06bfe7bfbdc072b4f3878d6c4598fc7b015d3 0B
warning: Layer size mismatch for sha256:6d4b4c91b3ff6970e22c704f59d45a5945065cb80bd7055114f6d27432e2205a: had 0, wrote 1251
warning: Layer size mismatch for sha256:8ea0315241b46a55ff650eddc8d06bfe7bfbdc072b4f3878d6c4598fc7b015d3: had 0, wrote 351520
warning: Layer size mismatch for sha256:59265c40e257554058624f35856dafd82d135c4ef406de298cb1fee647867381: had 0, wrote 2016712
warning: Layer size mismatch for sha256:80b1b554e5b4c4a4334fd25764d6c3d78bafd04ef727714dfb25b1356cc3a265: had 0, wrote 15521633
warning: Layer size mismatch for sha256:678f2cfebea627b8b3d5bed91a6fe1d3749421fc504a568a7977ce912a953763: had 0, wrote 17555180
warning: Layer size mismatch for sha256:e05d0d7eb99a1d91fa7349dd9a98a8b2d9e222b39fa305543e1ac4c380147a4b: had 0, wrote 17235178
sha256:ac5496c52004b990f981ee419eb338921753c35db97e382496b21f15c7d23792 file://coreos/etcd-operator
info: Mirroring completed in 43.13s (1.221MB/s)