d-v-b

zarr-metadata: v3 metadata as (document, context)

Status: design approved in conversation 2026-10-06; spec awaiting review. Builds on the 370→382 stack and the repair layer (d-v-b/zarr-python#391).

Problem

A v3 metadata model is read in a scope (Context): the definitions that decide what each extension name means. Today the model does not keep its scope. Each field (Read/Unclaimed) keeps the definition that read it, but

d-v-b

🤖 AI text below 🤖

zarr-python store API: the design gaps behind #4450, and archives versus stores

Written by an agent (Claude Code) at Davis Bennett's request, 2026-09-29, after seven adversarial review rounds of zarr-developers/zarr-python#4450. The archive/store distinction in the second half is Davis's.

The bugs fixed in #4450 come from five gaps in the Store base class, not from ZipStore alone. The 4.0 plan in zarr-python-planning already closes four of them for stateless stores, and keeps ZipStore as a writable, resource-holding exception; every ZipStore item still in the plan reproduces a bug the PR fixed. Distinguishing key-value archives, written whole, from key-value stores, which mutate individual keys, removes the exception: ZipStore becomes an archive with a pack() step, and the plan's open lifecycle question closes.

Bugs by root cause

d-v-b

# /// script
# requires-python = "<=3.13"
# dependencies = [
#   "zarr @ git+https://github.com/zarr-developers/zarr-python.git@main",
#   "numpy",
# ]
# ///

"""
The decoded memory layout is a runtime setting, not a property of the array.

d-v-b

#!/usr/bin/env python
# /// script
# requires-python = ">=3.11"
# dependencies = [
#     # FusedCodecPipeline is unreleased; install zarr pinned to the PR #3885
#     # merge commit. Swap this for "zarr>=3.2.0" once that release is on PyPI.
#     "zarr @ git+https://github.com/zarr-developers/zarr-python.git@fe50301564b6f5ca6552b5ca411a793682163760",
#     "numpy",
# ]
# ///

d-v-b

🤖 AI text below 🤖

AI contribution policies in open source: a survey

Originally surveyed on 2026-07-10 while rewriting zarr-python's contribution guidelines. The original research and prose were by Claude Code (claude-opus-4.8), directed and reviewed by Davis Bennett.

Corrected and updated on 2026-09-13 by Codex (GPT-6), at Davis Bennett's request. This revision is AI-written; the original human-review statement above does not attest to human review of this revision. The correction preserves the policy-design context while replacing unsupported generalizations and checking the comparisons below against primary sources.

Scope and corrections

d-v-b

Thin Metadata Refactor — Part 1: Upstream Metadata Models into zarr-metadata

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Add a zarr_metadata.model subpackage — frozen-dataclass models that are canonical, lossless representations of Zarr v2/v3 metadata documents — by porting the prototyped array models from ~/dev/zng/zng_metadata and adding new group/consolidated models, so a later zarr-python PR can consume them.

Architecture: zarr_metadata.model sits on top of the existing raw TypedDicts (ArrayMetadataV3, GroupMetadataV3, ConsolidatedMetadataV3, ZGroupMetadata, ...). Every v3 extension point is a ZarrMetadataV3 (name + configuration) value. Models do structural validation only; no codec/dtype/grid interpretation. One PR against d-v-b/zarr-python main.

Tech Stack: Python ≥3.11, frozen slott

d-v-b

Thin metadata refactor

Date: 2026-07-04 Status: approved design, pre-implementation

Goal

Narrow the scope of the array/group metadata classes to one job: forming a canonical, lossless representation of the JSON metadata documents. Everything the metadata classes currently do beyond that — instantiating codecs, chunk

d-v-b

ChunkSource: decoupling lazy views from their chunk source

Version: 1 Date: 2026-07-02 Status: design sketch (pre-implementation; nothing here blocks #3906)

Related:

• Lazy-view chunk layout design ("Layers A–C"): gist f50c93bc5f365e6986dadba6e4aa1902 — this doc is the natural "Layer D".
• Discussion #4082 (L0–L5 ladder; chunk_projections is L4).
• TensorStore drivers: stack, cast, virtual_chunked, downsample.

d-v-b

Design: represent zarr storage layout in encoding as a spec metadata fragment

Date: 2026-07-02 Status: approved (brainstorming), pending implementation plan Scope: xarray Zarr backend (xarray/backends/zarr.py)

Problem

The Zarr backend flattens a zarr array's metadata into individual .encoding keys (chunks, compressor/compressors, filters, serializer, shards,

d-v-b

Chunk layout & granularity for lazy array views

Version: 1 Date: 2026-07-01 Status: design (pending review) Context: the lazy-indexing branch (#3906) makes Array.lazy[...] return a zarr.Array whose _transform is non-identity — a view onto a subset of the backing array that generally does not tile evenly into the backing chunk grid.

Related: