Skip to content

Instantly share code, notes, and snippets.

View lmmx's full-sized avatar
💡
lights, camera, action

Louis Maddox lmmx

💡
lights, camera, action
View GitHub Profile
@lmmx
lmmx / generate_sample_data.py
Created September 8, 2025 12:43
Demo Avro schema inference and schema normalisation (array/map dtypes) from Avro schema itself - manually written SQL version: https://gist.github.com/lmmx/6296206ea3ad02473f05776c3b2ec284
import json
import random
# ISO-ish language codes to simulate
LANG_CODES = [
"en",
"fr",
"de",
"es",
"it",
@lmmx
lmmx / generate_sample_data.py
Last active September 8, 2025 12:44
Demo Avro schema inference and schema normalisation (array/map dtypes) - automatically generated SQL version: https://gist.github.com/lmmx/f6514291ed4fcd5399c021e40f6ffb21
import json
import random
# ISO-ish language codes to simulate
LANG_CODES = [
"en",
"fr",
"de",
"es",
"it",
@lmmx
lmmx / func_call_params_key.mmd
Created September 4, 2025 21:29
One of the nicer ways to display function call stack with parameters (albeit only for one parameter set per call)
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@lmmx
lmmx / demo.mmd
Last active September 4, 2025 18:28
Function call diagram for a couple of nested functions with many large parameters
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@lmmx
lmmx / mermaid_param_set_calls.mmd
Last active September 4, 2025 15:06
Demo of how to diagram calls by their param sets in Mermaid
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@lmmx
lmmx / string_format_email.mermaid
Created September 4, 2025 10:21
Trace of `cargo run -F cli tests/fixtures/jsonschema/string_format_email.json /tmp/out.avsc` rendered in Mermaid
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@lmmx
lmmx / generate_sample_data.py
Created August 29, 2025 02:11
Generating an Avro schema from a JSON schema (JSON schema made with genson-cli from generated NDJSON)
import json
examples = [
{
"id": "Q1",
"aliases": ["Universe", "Cosmos"], # list of strings
"labels": { # categorical map
"en": "universe",
"fr": "univers",
"de": "Universum"
import polars as pl
import fastavro
from pathlib import Path
# Define an Avro schema with both record (struct) and map
schema = {
"type": "record",
"name": "Inventory",
"fields": [
{"name": "id", "type": "int"},
@lmmx
lmmx / expand_trait.rs
Last active August 28, 2025 20:37
Diesel SQL compilation to string with variable substitution
#!/usr/bin/env rust-script
//! ```cargo
//! [dependencies]
//! diesel = { version = "2.2", features = ["sqlite"] }
//! diesel-dynamic-schema = "0.2"
//! rusqlite = "0.31"
//! serde = { version = "1", features = ["derive"] }
//! serde_json = "1"
//! ```
import duckdb
import polars as pl
from typing import Iterator, Optional
from polars.io.plugins import register_io_source
from duckdb import SQLExpression
import json
from decimal import Decimal
import datetime