Skip to content

Instantly share code, notes, and snippets.

View zeroshade's full-sized avatar

Matt Topol zeroshade

View GitHub Profile
@zeroshade
zeroshade / arrow_variant.md
Created April 29, 2025 22:28
Possible Arrow Variant extension proposal

Canonical Arrow Variant Extension Type

One important consideration across ALL of this: the fields for variant are supposed to be referenced by name not position which means that the names used metadata, value, and typed_value are REQUIRED to be utilized regardless of the rest of the schema utilized.

Unshredded (Easy Mode)

Let's first consider the simplest case of an unshredded variant, this is easy to model with Arrow via an extension type: