Skip to content

Instantly share code, notes, and snippets.

@x-yuri
Last active July 16, 2023 04:30
Show Gist options
  • Save x-yuri/c85c138a372390f5e7e9468056d6feeb to your computer and use it in GitHub Desktop.
Save x-yuri/c85c138a372390f5e7e9468056d6feeb to your computer and use it in GitHub Desktop.
Docker Hub digests

Docker Hub digests

There are non-multi-arch images (e.g. google/cloud-sdk). They're described by a manifest:

$ ./dhreq.sh google/cloud-sdk google/cloud-sdk/manifests/419.0.0-alpine -H 'Accept: application/vnd.docker.distribution.manifest.v2+json' -H 'Accept: application/vnd.docker.distribution.manifest.list.v2+json' | jq
< content-type: application/vnd.docker.distribution.manifest.v2+json
< docker-content-digest: sha256:e8c6e1e5831b850b33ae5454763300df0617bc8c758b7fcbb979e9e1d6ddb1f8
< etag: "sha256:e8c6e1e5831b850b33ae5454763300df0617bc8c758b7fcbb979e9e1d6ddb1f8"
JSON output (.config.digest == sha256:9580be7f2f08)
{
   "schemaVersion": 2,
   "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
   "config": {
      "mediaType": "application/vnd.docker.container.image.v1+json",
      "size": 4062,
      "digest": "sha256:9580be7f2f08a35fe3116fa7250c28c8235447914ee5dd311f1b9841925a47f3"
   },
   "layers": [
      {
         "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
         "size": 2826146,
         "digest": "sha256:895e193edb5191bf66fb5ccb29f5d3659e05eec5953255180cbdd66520e7c517"
      },
      {
         "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
         "size": 13982942,
         "digest": "sha256:7650e0398e0d20c2d572ee5f348adbe250e27e3984d3ddd0a0795c602f2a9f7c"
      },
      {
         "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
         "size": 1266,
         "digest": "sha256:bd9b35371b93717b8126648b1529e5f5a66f47abddd2ce071002d85abd4738ea"
      },
      {
         "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
         "size": 134,
         "digest": "sha256:b836fd40e151a79c0ef29f7260f38df25ad805af86e8eba22a6b09f10c436c3c"
      },
      {
         "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
         "size": 234887306,
         "digest": "sha256:96cac19032d792047f49ec705d9a75aacd2cf18130aff2666149fa5299fa50d0"
      },
      {
         "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
         "size": 195,
         "digest": "sha256:b61debef4c67274fd11680fe5cdf20530573e6dc66d57018efe92af3db26539a"
      }
   ]
}

And a config (used by the runtime to set up the container):

$ ./dhreq.sh google/cloud-sdk google/cloud-sdk/blobs/sha256:9580be7f2f08a35fe3116fa7250c28c8235447914ee5dd311f1b9841925a47f3 -H 'Accept: application/vnd.docker.container.image.v1+json' -L | jq
JSON output
{
  "architecture": "amd64",
  "config": {
    "Hostname": "",
    "Domainname": "",
    "User": "",
    "AttachStdin": false,
    "AttachStdout": false,
    "AttachStderr": false,
    "Tty": false,
    "OpenStdin": false,
    "StdinOnce": false,
    "Env": [
      "PATH=/google-cloud-sdk/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
      "CLOUD_SDK_VERSION=419.0.0"
    ],
    "Cmd": [
      "/bin/sh"
    ],
    "Image": "sha256:e0b0961a1137eac1eb4303f9635d9b9d476ca3db60e141c931d1b04658d61b55",
    "Volumes": {
      "/root/.config": {}
    },
    "WorkingDir": "",
    "Entrypoint": null,
    "OnBuild": null,
    "Labels": null
  },
  "container": "32cd822e3f8a547110cc8d89f169e9026bb608eb6e4fbb9f9c231ae577ab8c37",
  "container_config": {
    "Hostname": "32cd822e3f8a",
    "Domainname": "",
    "User": "",
    "AttachStdin": false,
    "AttachStdout": false,
    "AttachStderr": false,
    "Tty": false,
    "OpenStdin": false,
    "StdinOnce": false,
    "Env": [
      "PATH=/google-cloud-sdk/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
      "CLOUD_SDK_VERSION=419.0.0"
    ],
    "Cmd": [
      "/bin/sh",
      "-c",
      "#(nop) ",
      "VOLUME [/root/.config]"
    ],
    "Image": "sha256:e0b0961a1137eac1eb4303f9635d9b9d476ca3db60e141c931d1b04658d61b55",
    "Volumes": {
      "/root/.config": {}
    },
    "WorkingDir": "",
    "Entrypoint": null,
    "OnBuild": null,
    "Labels": {}
  },
  "created": "2023-02-22T15:54:24.737102903Z",
  "docker_version": "20.10.23",
  "history": [
    {
      "created": "2023-02-11T04:46:54.977923285Z",
      "created_by": "/bin/sh -c #(nop) ADD file:cdac18271416ac5bf6876b7ea9af1129108d03f9813589dfda113e5f09d6b80b in / "
    },
    {
      "created": "2023-02-11T04:46:55.092258571Z",
      "created_by": "/bin/sh -c #(nop)  CMD [\"/bin/sh\"]",
      "empty_layer": true
    },
    {
      "created": "2023-02-22T15:52:29.03624303Z",
      "created_by": "/bin/sh -c #(nop)  ARG CLOUD_SDK_VERSION=419.0.0",
      "empty_layer": true
    },
    {
      "created": "2023-02-22T15:52:29.226964309Z",
      "created_by": "/bin/sh -c #(nop)  ENV CLOUD_SDK_VERSION=419.0.0",
      "empty_layer": true
    },
    {
      "created": "2023-02-22T15:52:29.397593054Z",
      "created_by": "/bin/sh -c #(nop)  ENV PATH=/google-cloud-sdk/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
      "empty_layer": true
    },
    {
      "created": "2023-02-22T15:52:30.03973978Z",
      "created_by": "/bin/sh -c #(nop) COPY file:654d571d5fc2fd831c22967f637d09ac56724569abc67942fc70615a1b4500fb in /usr/local/bin/docker "
    },
    {
      "created": "2023-02-22T15:52:31.423030824Z",
      "created_by": "/bin/sh -c addgroup -g 1000 -S cloudsdk &&     adduser -u 1000 -S cloudsdk -G cloudsdk"
    },
    {
      "created": "2023-02-22T15:52:33.270683053Z",
      "created_by": "/bin/sh -c if [ `uname -m` = 'x86_64' ]; then echo -n \"x86_64\" > /tmp/arch; else echo -n \"arm\" > /tmp/arch; fi;"
    },
    {
      "created": "2023-02-22T15:53:59.868121909Z",
      "created_by": "/bin/sh -c ARCH=`cat /tmp/arch` && apk --no-cache add         curl         python3         py3-crcmod         py3-openssl         bash         libc6-compat         openssh-client         git         gnupg     && curl -O https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-cli-${CLOUD_SDK_VERSION}-linux-${ARCH}.tar.gz &&     tar xzf google-cloud-cli-${CLOUD_SDK_VERSION}-linux-${ARCH}.tar.gz &&     rm google-cloud-cli-${CLOUD_SDK_VERSION}-linux-${ARCH}.tar.gz &&     gcloud config set core/disable_usage_reporting true &&     gcloud config set component_manager/disable_update_check true &&     gcloud config set metrics/environment github_docker_image &&     gcloud --version"
    },
    {
      "created": "2023-02-22T15:54:23.920281695Z",
      "created_by": "/bin/sh -c git config --system credential.'https://source.developers.google.com'.helper gcloud.sh"
    },
    {
      "created": "2023-02-22T15:54:24.737102903Z",
      "created_by": "/bin/sh -c #(nop)  VOLUME [/root/.config]",
      "empty_layer": true
    }
  ],
  "os": "linux",
  "rootfs": {
    "type": "layers",
    "diff_ids": [
      "sha256:4e64766527982cca5b986a8d061530bf5c32e111047733b59cb0c9742a89eda0",
      "sha256:e1e1e04c1c51b9764fc062d27169876128827a193fe78187b032018a7aa5f6ed",
      "sha256:501951f9e0eb81bd3d193c9be60c3abbba2876af54730bbe706724e31726f192",
      "sha256:c841c0c38550fa32d6c67718f576bfcfebfda78c6df7e725411970ce3a011ff7",
      "sha256:42ff958a524d61d1e36a69d32e6be48f8f2997a97e4fab5e3a7067ba201d65e5",
      "sha256:b92be71a568d923e0e259e07f49a76f3180275766c2efbe6093c50b337804570"
    ]
  }
}
$ docker images | grep ^google/cloud-sdk
google/cloud-sdk  419.0.0-alpine         9580be7f2f08   3 months ago    922MB
$ docker inspect 9580be7f2f08 --format '{{json .}}' | jq '. | {Id, RepoDigests}'
{
  "Id": "sha256:9580be7f2f08a35fe3116fa7250c28c8235447914ee5dd311f1b9841925a47f3",
  "RepoDigests": [
    "google/cloud-sdk@sha256:e8c6e1e5831b850b33ae5454763300df0617bc8c758b7fcbb979e9e1d6ddb1f8"
  ]
}

Image id (9580be7f2f08) is the config digest. RepoDigest (e8c6e1e5831b) is the digest of the manifest and that is what you can see on Docker Hub, or in the Docker-Content-Digest/ETag headers of the manifest.

There are multi-arch images (e.g. alpine). They're are described by a manifest list:

$ ./dhreq.sh alpine library/alpine/manifests/3.18.2 -H 'Accept: application/vnd.docker.distribution.manifest.v2+json' -H 'Accept: application/vnd.docker.distribution.manifest.list.v2+json' | jq
< content-type: application/vnd.docker.distribution.manifest.list.v2+json
< docker-content-digest: sha256:82d1e9d7ed48a7523bdebc18cf6290bdb97b82302a8a9c27d4fe885949ea94d1
< etag: "sha256:82d1e9d7ed48a7523bdebc18cf6290bdb97b82302a8a9c27d4fe885949ea94d1"
JSON output (.manifests.0.digest == sha256:25fad2a32ad1)
{
  "manifests": [
    {
      "digest": "sha256:25fad2a32ad1f6f510e528448ae1ec69a28ef81916a004d3629874104f8a7f70",
      "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
      "platform": {
        "architecture": "amd64",
        "os": "linux"
      },
      "size": 528
    },
    {
      "digest": "sha256:ae30c2911284159e0dc2f244b5e7a8b801b9c9f3449806d6e5591de22b65ce15",
      "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
      "platform": {
        "architecture": "arm",
        "os": "linux",
        "variant": "v6"
      },
      "size": 528
    },
    {
      "digest": "sha256:0b75b5bfd67c3ffaee0e951533407f6d45d53d7f4dd139fa0c09747b4849dd5d",
      "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
      "platform": {
        "architecture": "arm",
        "os": "linux",
        "variant": "v7"
      },
      "size": 528
    },
    {
      "digest": "sha256:e3bd82196e98898cae9fe7fbfd6e2436530485974dc4fb3b7ddb69134eda2407",
      "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
      "platform": {
        "architecture": "arm64",
        "os": "linux",
        "variant": "v8"
      },
      "size": 528
    },
    {
      "digest": "sha256:bd649691cf299c58fec56fb84a5067a915da6915897c6f846a6e317e5ff42a4d",
      "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
      "platform": {
        "architecture": "386",
        "os": "linux"
      },
      "size": 528
    },
    {
      "digest": "sha256:8d42f68528a085fe2d936dcca64c642463744eb47312bb8e95863464550165ca",
      "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
      "platform": {
        "architecture": "ppc64le",
        "os": "linux"
      },
      "size": 528
    },
    {
      "digest": "sha256:579fb3e58c23e1dba58ce7d06a14417954d0daaca4e28fa0358e941895d752f8",
      "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
      "platform": {
        "architecture": "s390x",
        "os": "linux"
      },
      "size": 528
    }
  ],
  "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
  "schemaVersion": 2
}
$ ./dhreq.sh alpine library/alpine/manifests/sha256:25fad2a32ad1f6f510e528448ae1ec69a28ef81916a004d3629874104f8a7f70 -H 'Accept: application/vnd.docker.distribution.manifest.v2+json' -H 'Accept: application/vnd.docker.distribution.manifest.list.v2+json' | jq
< content-type: application/vnd.docker.distribution.manifest.v2+json
< docker-content-digest: sha256:25fad2a32ad1f6f510e528448ae1ec69a28ef81916a004d3629874104f8a7f70
< etag: "sha256:25fad2a32ad1f6f510e528448ae1ec69a28ef81916a004d3629874104f8a7f70"
JSON output (.config.digest == sha256:c1aabb73d233)
{
  "schemaVersion": 2,
  "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
  "config": {
    "mediaType": "application/vnd.docker.container.image.v1+json",
    "size": 1472,
    "digest": "sha256:c1aabb73d2339c5ebaa3681de2e9d9c18d57485045a4e311d9f8004bec208d67"
  },
  "layers": [
    {
      "mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",
      "size": 3397879,
      "digest": "sha256:31e352740f534f9ad170f75378a84fe453d6156e40700b882d737a8f4a6988a3"
    }
  ]
}
$ ./dhreq.sh alpine library/alpine/blobs/sha256:c1aabb73d2339c5ebaa3681de2e9d9c18d57485045a4e311d9f8004bec208d67 -H 'Accept: application/vnd.docker.container.image.v1+json' -L | jq
JSON output
{
  "architecture": "amd64",
  "config": {
    "Hostname": "",
    "Domainname": "",
    "User": "",
    "AttachStdin": false,
    "AttachStdout": false,
    "AttachStderr": false,
    "Tty": false,
    "OpenStdin": false,
    "StdinOnce": false,
    "Env": [
      "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
    ],
    "Cmd": [
      "/bin/sh"
    ],
    "Image": "sha256:5b8658701c96acefe1cd3a21b2a80220badf9124891ad440d95a7fa500d48765",
    "Volumes": null,
    "WorkingDir": "",
    "Entrypoint": null,
    "OnBuild": null,
    "Labels": null
  },
  "container": "bfc8078c169637d70e40ce591b5c2fe8d26329918dafcb96ebc9304ddff162ea",
  "container_config": {
    "Hostname": "bfc8078c1696",
    "Domainname": "",
    "User": "",
    "AttachStdin": false,
    "AttachStdout": false,
    "AttachStderr": false,
    "Tty": false,
    "OpenStdin": false,
    "StdinOnce": false,
    "Env": [
      "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
    ],
    "Cmd": [
      "/bin/sh",
      "-c",
      "#(nop) ",
      "CMD [\"/bin/sh\"]"
    ],
    "Image": "sha256:5b8658701c96acefe1cd3a21b2a80220badf9124891ad440d95a7fa500d48765",
    "Volumes": null,
    "WorkingDir": "",
    "Entrypoint": null,
    "OnBuild": null,
    "Labels": {}
  },
  "created": "2023-06-14T20:41:59.079795125Z",
  "docker_version": "20.10.23",
  "history": [
    {
      "created": "2023-06-14T20:41:58.950178204Z",
      "created_by": "/bin/sh -c #(nop) ADD file:1da756d12551a0e3e793e02ef87432d69d4968937bd11bed0af215db19dd94cd in / "
    },
    {
      "created": "2023-06-14T20:41:59.079795125Z",
      "created_by": "/bin/sh -c #(nop)  CMD [\"/bin/sh\"]",
      "empty_layer": true
    }
  ],
  "os": "linux",
  "rootfs": {
    "type": "layers",
    "diff_ids": [
      "sha256:78a822fe2a2d2c84f3de4a403188c45f623017d6a4521d23047c9fbb0801794c"
    ]
  }
}
$ docker images | grep ^alpine
alpine  3.18                   c1aabb73d233   6 days ago      7.33MB
$ docker inspect c1aabb73d233 --format '{{json .}}' | jq '. | {Id, RepoDigests}'
{
  "Id": "sha256:c1aabb73d2339c5ebaa3681de2e9d9c18d57485045a4e311d9f8004bec208d67",
  "RepoDigests": [
    "alpine@sha256:82d1e9d7ed48a7523bdebc18cf6290bdb97b82302a8a9c27d4fe885949ea94d1"
  ]
}

Image id (c1aabb73d233) is the config digest. RepoDigest (82d1e9d7ed48) is the digest of the manifest list. And on Docker Hub you can see the manifest digests, which you can find in the manifest list, or in the Docker-Content-Digest/ETag headers of the manifest (e.g. 25fad2a32ad1).

That is, Docker Hub shows manifest digests. You can find it in RepoDigest for non-multi-arch images. For multi-arch images you need the manifest list to figure it out:

$ ./dhreq.sh alpine library/alpine/manifests/3.18.2 -H 'Accept: application/vnd.docker.distribution.manifest.list.v2+json'

dhreq.sh:

#!/bin/sh -eu
image=$1
req=$2
shift 2
if [ "`expr index "$image" /`" = 0 ]; then
    image=library/$image
fi
token=`curl -sS \
        "https://auth.docker.io/token?service=registry.docker.io&scope=repository:$image:pull" \
    | jq -r .token`
curl -sS "https://registry-1.docker.io/v2/$req" \
    -H "Authorization: Bearer $token" \
    "$@"

To make dhreq.sh more robust see this answer.

Image manifests describe the various constituents of a docker image.

https://docs.docker.com/registry/spec/manifest-v2-1/

The image manifest provides a configuration and a set of layers for a container image.

The config field references a configuration object for a container, by digest. This configuration item is a JSON blob that the runtime uses to set up the container.

When pulling images, clients indicate support for this new version of the manifest format by sending the application/vnd.docker.distribution.manifest.v2+json and application/vnd.docker.distribution.manifest.list.v2+json media types in an Accept header when making a request to the manifests endpoint. Updated clients should check the Content-Type header to see whether the manifest returned from the endpoint is in the old format, or is an image manifest or manifest list in the new format.

If the manifest being requested uses the new format, and the appropriate media type is not present in an Accept header, the registry will assume that the client cannot handle the manifest as-is, and rewrite it on the fly into the old format. If the object that would otherwise be returned is a manifest list, the registry will look up the appropriate manifest for the amd64 platform and linux OS, rewrite that manifest into the old format if necessary, and return the result to the client. If no suitable manifest is found in the manifest list, the registry will return a 404 error.

https://docs.docker.com/registry/spec/manifest-v2-2/

The client should include an Accept header indicating which manifest content types it supports. For more details on the manifest formats and their content types, see manifest-v2-1.md and manifest-v2-2.md.

To provide verification of http content, any response may include a Docker-Content-Digest header. This will include the digest of the target entity returned in the response.

https://docs.docker.com/registry/spec/api/

The image manifest file will contain all the information which is needed to pull, install, validate and run an image. It will contain a list of layers by a content addressable id, history, run time configuration, and signatures.

moby/moby#8093

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment