Skip to content

Instantly share code, notes, and snippets.

View lmmx's full-sized avatar
💡
lights, camera, action

Louis Maddox lmmx

💡
lights, camera, action
View GitHub Profile
@lmmx
lmmx / generate_sample_data.py
Created August 29, 2025 02:11
Generating an Avro schema from a JSON schema (JSON schema made with genson-cli from generated NDJSON)
import json
examples = [
{
"id": "Q1",
"aliases": ["Universe", "Cosmos"], # list of strings
"labels": { # categorical map
"en": "universe",
"fr": "univers",
"de": "Universum"
import polars as pl
import fastavro
from pathlib import Path
# Define an Avro schema with both record (struct) and map
schema = {
"type": "record",
"name": "Inventory",
"fields": [
{"name": "id", "type": "int"},
@lmmx
lmmx / expand_trait.rs
Last active August 28, 2025 20:37
Diesel SQL compilation to string with variable substitution
#!/usr/bin/env rust-script
//! ```cargo
//! [dependencies]
//! diesel = { version = "2.2", features = ["sqlite"] }
//! diesel-dynamic-schema = "0.2"
//! rusqlite = "0.31"
//! serde = { version = "1", features = ["derive"] }
//! serde_json = "1"
//! ```
import duckdb
import polars as pl
from typing import Iterator, Optional
from polars.io.plugins import register_io_source
from duckdb import SQLExpression
import json
from decimal import Decimal
import datetime
@lmmx
lmmx / clone_gh_labels.sh
Created August 26, 2025 11:43
`gh labels` repo clone script: delete the default ones and clone some good ones (from https://github.com/lmmx/polars-diskcache/labels)
# Delete all the unwanted defaults
gh label delete bug --yes
gh label delete documentation --yes
gh label delete duplicate --yes
gh label delete enhancement --yes
gh label delete "good first issue" --yes
gh label delete "help wanted" --yes
gh label delete invalid --yes
gh label delete question --yes
gh label delete wontfix --yes
@lmmx
lmmx / hassemap.rs
Created August 23, 2025 11:07
HasseMap: poset key-value map maintaining the partial order of observed keys, e.g. for JSON rows with each entry having column subsets of a total column schema
#!/usr/bin/env rust-script
//! ```cargo
//! [dependencies]
//! ordermap = "0.5"
//! serde = { version = "1", features = ["derive"] }
//! serde_json = { version = "1", features = ["preserve_order"] }
//! anyhow = "1"
//! ```
use anyhow::Result;
If you debug the Python stack trace when passing in a simple 2 col DF
```py
(polars-genson-py) louis 🌟 ~/dev/polars-genson/polars-genson-py $ qp schema_test.py
[1] > /home/louis/dev/polars-genson/polars-genson-py/python/polars_genson/__init__.py(45)schema_to_json()
-> breakpoint()
(Pdb++) n
[1] > /home/louis/dev/polars-genson/polars-genson-py/python/polars_genson/__init__.py(46)schema_to_json()
-> return _rust_schema_to_json(df)
(Pdb++) s
@lmmx
lmmx / Justfile
Created August 21, 2025 18:23
select menu for conventional commit
# Git status report task
gnews:
#!/usr/bin/env python3
from subprocess import check_output
gnew = check_output(["git", "status", "--porcelain"], text=True).splitlines()
groups = {
"M": "Changed",
"A": "Added",
@lmmx
lmmx / Justfile
Created August 21, 2025 17:36
Git commit message automation Justfile task
# Git status report task
gnews:
#!/usr/bin/env python3
from subprocess import check_output
gnew = check_output(["git", "status", "--porcelain"], text=True).splitlines()
groups = {
"M": "Changed",
"A": "Added",
@lmmx
lmmx / Justfile
Created August 21, 2025 15:43
Demoing that a bash `trap` EXIT statement is scoped to the end of a Justfile task (recipe) block
main:
just trap-demo
just do-more-things
echo ">>> Finished"
trap-demo:
#!/usr/bin/env bash
set -e
echo ">>> Task starting"