Skip to content

Instantly share code, notes, and snippets.

@lmolkova

lmolkova/acr_11_ga.md

Last active March 15, 2023 18:51
Show Gist options
  • Save lmolkova/5dee2773dd3e1d61f73a41153d8a28e1 to your computer and use it in GitHub Desktop.
Save lmolkova/5dee2773dd3e1d61f73a41153d8a28e1 to your computer and use it in GitHub Desktop.
Download ZIP
Azure Container Registry 1.1.0 SDK GA
Raw
acr_11_ga.md

Champion Scenarios in Container Registry 1.1.0 SDKs

  • Upload/download OCI images
  • Upload/download custom manifest
  • Delete manifests and blobs

Java

Create blob client

New scenarios are implemented in a different client - ContainerRegistryBlobClient in specialized namespace.

import com.azure.containers.containerregistry.specialized.ContainerRegistryBlobAsyncClient;
import com.azure.containers.containerregistry.specialized.ContainerRegistryBlobClientBuilder;

...

ContainerRegistryBlobAsyncClient blobClient = new ContainerRegistryBlobClientBuilder()
    .endpoint(ENDPOINT)
    .repository(REPOSITORY)
    .credential(credential)
    .buildAsyncClient();

Upload an OCI Image (sync)

// Upload config - it comes from user models and can be anything
BinaryData configContent = BinaryData.fromString("{}");

// simple upload from binary data
UploadBlobResult configUploadResult = blobClient.uploadBlob(configContent);

OciDescriptor configDescriptor = new OciDescriptor()
    .setMediaType("application/vnd.unknown.config.v1+json") // we do not provide content types for configs - it's an open set
    .setDigest(configUploadResult.getDigest())
    .setSizeInBytes(configContent.getLength());

// Upload layers one by one from files
try (FileInputStream content = new FileInputStream("artifact.v5.tar.gz")) {
    UploadBlobResult layerUploadResult = blobClient.uploadBlob(content.getChannel(), Context.NONE);

    // now we can create manifest from config and layers
    OciImageManifest manifest = new OciImageManifest()
        .setConfig(configDescriptor)
        .setLayers(Collections.singletonList(
            new OciDescriptor()
                .setDigest(layerUploadResult.getDigest())
                .setSizeInBytes(layerUploadResult.getSizeInBytes())
                .setMediaType("application/vnd.oci.image.layer.v1.tar+gzip")));

    UploadManifestResult manifestResult = blobClient.uploadManifest(manifest, "v5");
}

Upload blob from file async

Flux<ByteBuffer> layerContent = Flux.using(
    () -> new FileInputStream("artifact.v5.tar.gz"),
    fileStream -> blobClient.uploadBlob(FluxUtil.toFluxByteBuffer(fileStream, CHUNK_SIZE)),
    this::closeStream);

Download an OCI Image (sync)

DownloadManifestResult manifestResult = blobClient.downloadManifest("latest");

OciImageManifest manifest = manifestResult.asOciManifest();

String configFileName = manifest.getConfig().getDigest() + ".json";
blobClient.downloadStream(manifest.getConfig().getDigest(), createFileChannel(configFileName));

for (OciDescriptor layer : manifest.getLayers()) {
    blobClient.downloadStream(layer.getDigest(), createFileChannel(layer.getDigest()));
}

// ....

private static SeekableByteChannel createFileChannel(String fileName) throws IOException {
    fileName = trimSha(fileName);
    return Files.newByteChannel(Paths.get(OUT_DIRECTORY, fileName), StandardOpenOption.CREATE_NEW, StandardOpenOption.WRITE);
}

Download blob (async)

Async download returns DownloadBlobAsyncResult and content can be sent to socket, file, in a different operation.

blobClient
    .downloadStream(digest)
    .flatMap(downloadResult ->
        Mono.using(
            () -> openSocket(),
            socket -> downloadResult.writeValueToAsync(socket),
            socket -> close(socket)))
    .block();
// ....

private static AsynchronousSocketChannel openSocket() {
    return new AsynchronousSocketChannel(...).bind(...);
}

Upload an OCI Image with custom manifest type

String manifest = """
{
    "schemaVersion": 2,
    "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
    "manifests": [
    {
        "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
        "digest": "sha256:e692418e4cbaf90ca69d05a66403747baa33ee08806650b51fab815ad7fc331f",
        "size": 7143,
        "platform": {
        "architecture": "ppc64le",
        "os": "linux"
        }
    },
    {
        "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
        "digest": "sha256:5b0bcabd1ed22e9fb1310cf6c2dec7cdef19f0ad69efa1f392e94a4333501270",
        "size": 7682,
        "platform": {
        "architecture": "amd64",
        "os": "linux",
        "features": ["sse4"]
        }
    }
    ]
}""";

ManifestMediaType manifestListType = 
    ManifestMediaType.fromString("application/vnd.docker.distribution.manifest.list.v2+json");
BinaryData manifestList = BinaryData.fromString(manifest);

Response<UploadManifestResult> result = blobClient.uploadManifestWithResponse(new UploadManifestOptions(manifestList, manifestListType));

Download an OCI Image with custom manifest type

ManifestMediaType manifestListType = ManifestMediaType.fromString("application/vnd.docker.distribution.manifest.list.v2+json");
ManifestMediaType ociIndexType = ManifestMediaType.fromString("application/vnd.oci.image.index.v1+json");

blobClient
    .downloadManifestWithResponse("latest", Arrays.asList(manifestListType, ociIndexType))
    .doOnNext(downloadResult -> {
        if (manifestListType.equals(downloadResult.getValue().getMediaType())) {
            DockerManifestList manifestList = downloadResult.getValue().getContent().toObject(DockerManifestList.class);

        } else if (ociIndexType.equals(downloadResult.getValue().getMediaType())) {
            OciIndex ociIndex = downloadResult.getValue().getContent().toObject(OciIndex.class);

        } else {
            throw new IllegalArgumentException("Got unexpected content type: " + downloadResult.getValue().getMediaType());
        }
    })
    .block();

Delete blob (sync)

DownloadManifestResult manifestResult = blobClient.downloadManifest("latest");

for (OciDescriptor layer : manifestResult.asOciManifest().getLayers()) {
    blobClient.deleteBlob(layer.getDigest());
}

Delete manifest (async)

blobClient.downloadManifest("latest")
    .flatMap(manifest -> blobClient.deleteManifest(manifest.getDigest()))
    .block();

NET

Source

Create blob client

// Get the service endpoint from the environment
Uri endpoint = new Uri(Environment.GetEnvironmentVariable("REGISTRY_ENDPOINT"));

string repository = "sample-oci-image";
string tag = "demo";

// Create a new ContainerRegistryBlobClient
ContainerRegistryBlobClient client = new(endpoint, repository, new DefaultAzureCredential(), new ContainerRegistryClientOptions());

Upload an OCI Image

// Create a manifest to list files in this image
OciImageManifest manifest = new();

// Upload a config
BinaryData config = BinaryData.FromString("Sample config");
UploadBlobResult uploadConfigResult = await client.UploadBlobAsync(config);

// Update manifest with config info
manifest.Config = new OciDescriptor()
{
    Digest = uploadConfigResult.Digest,
    SizeInBytes = uploadConfigResult.SizeInBytes,
    MediaType = "application/vnd.oci.image.config.v1+json"
};

// Upload a layer file
using Stream stream = File.OpenRead(Path.Combine(path, uploadFileName));
UploadBlobResult uploadLayerResult = await client.UploadBlobAsync(stream);

// Update manifest with layer info
manifest.Layers.Add(new OciDescriptor()
{
    Digest = uploadLayerResult.Digest,
    SizeInBytes = uploadLayerResult.SizeInBytes,
    MediaType = "application/vnd.oci.image.layer.v1.tar"
});

// Finally, upload the manifest file
await client.UploadManifestAsync(manifest, tag);

Download an OCI Image

// Download the manifest to obtain the list of files in the image
DownloadManifestResult result = await client.DownloadManifestAsync(tag);
OciImageManifest manifest = result.AsOciManifest();

string manifestFile = Path.Combine(path, "manifest.json");
using (FileStream stream = File.Create(manifestFile))
{
    await result.Content.ToStream().CopyToAsync(stream);
}

// Download and write out the config
DownloadBlobResult configBlob = await client.DownloadBlobAsync(manifest.Config.Digest);

string configFile = Path.Combine(path, "config.json");
using (FileStream stream = File.Create(configFile))
{
    await configBlob.Content.ToStream().CopyToAsync(stream);
}

// Download and write out the layers
foreach (OciDescriptor layerInfo in manifest.Layers)
{
    string layerFile = Path.Combine(path, TrimSha(layerInfo.Digest));
    using (FileStream stream = File.Create(layerFile))
    {
        await client.DownloadBlobToAsync(layerInfo.Digest, stream);
    }
}

Upload an OCI Image with custom manifest type

// Create a manifest file in the Docker v2 Manifest List format
var manifestList = new
{
    mediaType = ManifestMediaType.DockerManifestList.ToString(),
    manifests = new[]
    {
        new
        {
            digest = "sha256:f54a58bc1aac5ea1a25d796ae155dc228b3f0e11d046ae276b39c4bf2f13d8c4",
            mediaType = ManifestMediaType.DockerManifest.ToString(),
            platform = new {
                architecture = ArtifactArchitecture.Amd64.ToString(),
                os = ArtifactOperatingSystem.Linux.ToString()
            }
        }
    }
};

// Finally, upload the manifest file
BinaryData content = BinaryData.FromObjectAsJson(manifestList);
await client.UploadManifestAsync(content, tag: "sample", ManifestMediaType.DockerManifestList);

Download an OCI Image with custom manifest type

// Pass multiple media types if the media type of the manifest to download is unknown
List<ManifestMediaType> mediaTypes = new() {
    "application/vnd.docker.distribution.manifest.list.v2+json",
    "application/vnd.oci.image.index.v1+json" };

DownloadManifestResult result = await client.DownloadManifestAsync("sample", mediaTypes);

if (result.MediaType == "application/vnd.docker.distribution.manifest.list.v2+json")
{
    Console.WriteLine("Manifest is a Docker manifest list.");
}
else if (result.MediaType == "application/vnd.oci.image.index.v1+json")
{
    Console.WriteLine("Manifest is an OCI index.");
}

Delete blob

DownloadManifestResult result = await client.DownloadManifestAsync(tag);
OciImageManifest manifest = result.AsOciManifest();

foreach (OciDescriptor layerInfo in manifest.Layers)
{
    await client.DeleteBlobAsync(layerInfo.Digest);
}

Delete manifest

DownloadManifestResult downloadManifestResult = await client.DownloadManifestAsync(tag);
await client.DeleteManifestAsync(downloadManifestResult.Digest);

JavaScript

Source

Create client

const {
  ContainerRegistryBlobClient,
  KnownContainerRegistryAudience,
} = require("@azure/container-registry");

...

const client = new ContainerRegistryBlobClient(
endpoint,
repository,
new DefaultAzureCredential());

Upload an OCI Image

const config = Buffer.from("Sample config");
const { digest: configDigest, sizeInBytes: configSize } = await client.uploadBlob(config);

const layer = Buffer.from("Hello, world");
const { digest: layerDigest, sizeInBytes: layerSize } = await client.uploadBlob(layer);

const manifest: OciImageManifest = {
  config: {
    digest: configDigest,
    sizeInBytes: configSize,
    mediaType: "application/vnd.oci.image.config.v1+json",
  },
  layers: [
    {
      digest: layerDigest,
      sizeInBytes: layerSize,
      mediaType: "application/vnd.oci.image.layer.v1.tar",
    },
  ],
};

await client.uploadManifest(manifest, { tag: "demo" });

Download an OCI Image

// Download the manifest to obtain the list of files in the image based on the tag
const result = await client.downloadManifest("demo");

// If an OCI image manifest was downloaded, it is available as a strongly typed object via the `manifest` property.
if (!isDownloadOciImageManifestResult(result)) {
  throw new Error("Expected an OCI image manifest");
}

const manifest = result.manifest;
// Manifests of all media types can be written to a file using the `content` stream.
const manifestFile = fs.createWriteStream("manifest.json");
result.content.pipe(manifestFile);

const configResult = await client.downloadBlob(manifest.config.digest);
const configFile = fs.createWriteStream("config.json");
configResult.content.pipe(configFile);

// Download and write out the layers
for (const layer of manifest.layers) {
  const fileName = trimSha(layer.digest);
  const layerStream = fs.createWriteStream(fileName);
  const downloadLayerResult = await client.downloadBlob(layer.digest);
  downloadLayerResult.content.pipe(layerStream);
}

Upload an OCI Image with custom manifest type

const mediaType = "application/vnd.docker.distribution.manifest.list.v2+json";

const manifest = Buffer.from(
  JSON.stringify({
    schemaVersion: 2,
    mediaType,
    manifests: [
      {
        mediaType: "application/vnd.docker.distribution.manifest.v2+json",
        digest: "sha256:e692418e4cbaf90ca69d05a66403747baa33ee08806650b51fab815ad7fc331f",
        size: 7143,
        platform: {
          architecture: "ppc64le",
          os: "linux",
        },
      },
      {
        mediaType: "application/vnd.docker.distribution.manifest.v2+json",
        digest: "sha256:5b0bcabd1ed22e9fb1310cf6c2dec7cdef19f0ad69efa1f392e94a4333501270",
        size: 7682,
        platform: {
          architecture: "amd64",
          os: "linux",
          features: ["sse4"],
        },
      },
    ],
  })
);

await client.uploadManifest(manifest, { mediaType });

Download an OCI Image with custom manifest type

const manifestListType = "application/vnd.docker.distribution.manifest.list.v2+json";
const ociIndexType = "application/vnd.oci.image.index.v1+json";

const result = await client.downloadManifest("latest", {
  mediaType: [manifestListType, ociIndexType],
});

if (result.mediaType === manifestListType) {
  console.log("Manifest is a Docker manifest list");
} else if (result.mediaType === ociIndexType) {
  console.log("Manifest is an OCI index");
}

Delete blob

const downloadResult = await client.downloadManifest("latest");

for (const layer of downloadResult.manifest.layers) {
  await client.deleteBlob(layer.digest);
}

Delete manifest

const downloadResult = await client.downloadManifest("latest");
await client.deleteManifest(downloadResult.digest);

Python

Sources

Create blob client

from azure.containerregistry import ContainerRegistryClient
...
with ContainerRegistryClient(self.endpoint, self.credential, audience=self.audience) as client

Upload an OCI Image

layer = BytesIO(b"Sample layer")
config = BytesIO(json.dumps(
{
    "sample config": "content",
}).encode())
    
layer_digest, layer_size = await client.upload_blob(repository_name, layer)
# Upload a config
config_digest, config_size = await client.upload_blob(repository_name, config)
# Create a manifest with config and layer info
manifest = OCIManifest(
    config = Descriptor(
        media_type="application/vnd.oci.image.config.v1+json",
        digest=config_digest,
        size=config_size
    ),
    layers=[
        Descriptor(
            media_type="application/vnd.oci.image.layer.v1.tar",
            digest=layer_digest,
            size=layer_size,
            annotations=Annotations(name="artifact.txt")
        )
    ]
)
# Upload the manifest
digest = await client.upload_manifest(repository_name, manifest)

Download an OCI Image

download_manifest_result = await client.download_manifest(repository_name, digest)
downloaded_manifest = download_manifest_result.manifest
download_manifest_stream = download_manifest_result.data

print(b"".join(download_manifest_stream))

# Download the layers
async for layer in downloaded_manifest.layers:
    async with await client.download_blob(repository_name, layer.digest) as layer_stream:
        print(b"".join(layer_stream))

# Download the config
async with await client.download_blob(repository_name, downloaded_manifest.config.digest) as config_stream:
    print(b"".join(config_stream))

Download to file:

for layer in downloaded_manifest.layers:
    try:
        with client.download_blob(repository_name, layer.digest) as stream:
            with open("layer_name", "wb") as layer_file:
                for data in stream:
                    layer_file.write(data)
    except ValueError:
        # Downloaded digest value did not match. Deleting file
        os.remove("layer_file")

Upload an OCI Image with custom manifest type

// create a Docker manifest object in Docker v2 Manifest List format
manifest_list = {
    "schemaVersion": 2,
    "mediaType": ManifestMediaType.DOCKER_MANIFEST_LIST,
    "manifests": [
        {
            "digest": "sha256:f54a58bc1aac5ea1a25d796ae155dc228b3f0e11d046ae276b39c4bf2f13d8c4",
            "mediaType": ManifestMediaType.DOCKER_MANIFEST,
            "platform": {
                "architecture": ArtifactArchitecture.AMD64,
                "os": ArtifactOperatingSystem.LINUX
            }
        }
    ]
}
manifest_bytes = json.dumps(manifest_list).encode()
...
# Upload the manifest with one custom media type
async client.upload_manifest(self.repository_name, BytesIO(manifest_bytes), tag="sample", media_type=ManifestMediaType.DOCKER_MANIFEST_LIST)

Download an OCI Image with custom manifest type

# Download the manifest with multiple custom media types
download_manifest_result = async client.download_manifest(self.repository_name, "sample", media_types=[ManifestMediaType.DOCKER_MANIFEST_LIST, ManifestMediaType.OCI_INDEX])
download_manifest_stream = download_manifest_result.data
download_manifest_media_type = download_manifest_result.media_type
print(b"".join(download_manifest_stream))
print(download_manifest_media_type)

Delete blob

async for layer in downloaded_manifest.layers:
    await client.delete_blob(repository_name, layer.digest)

Delete manifest

await client.delete_manifest(repository_name, download_manifest_result.digest)

Swagger APIs

Docker V2 APIs to upload/download blobs and manifests (stable 2021-07-01):

Open questions

  • Download stream vs chunks:
    • two APIs in Python and JS?
    • return response?
  • Specialized, blob client, multiple builders
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment