Skip to content

Instantly share code, notes, and snippets.

View marklit's full-sized avatar

Mark Litwintschik marklit

View GitHub Profile
@marklit
marklit / issue.md
Created July 21, 2024 17:46
Overture July 2024 Canadian Addresses

DuckDB v1.0.0 1f98600c2c

$ cd '/mnt/j/gis/Global/overture/2024_07/theme=addresses/type=address'
$ COUNTRY=CA
$ echo "COPY (
              SELECT h3_cell_to_boundary_wkt(
                          h3_latlng_to_cell(bbox.ymax, bbox.xmax, 5))::geometry geom,
                      COUNT(*)
 FROM read_parquet('*.parquet')
COPY (
SELECT * EXCLUDE (geometry,
bbox,
names,
sources),
ST_GEOMFROMWKB(geometry) geom,
JSON(bbox) AS bbox,
JSON(names) AS names,
JSON(sources) AS sources
FROM read_parquet('s3://overturemaps-us-west-2/release/2024-06-13-beta.0/theme=buildings/type=building/*00208*',
@marklit
marklit / satellogic-samples.md
Last active July 5, 2024 09:07
Satellogic Samples

Signature version complaint via awscli

$ virtualenv ~/.aws
$ source ~/.aws/bin/activate
$ pip install awscli

$ aws --version
@marklit
marklit / highlights.py
Created May 28, 2024 05:49
More Itertools Highlights
# https://more-itertools.readthedocs.io/en/stable/index.html
list(chunked([1, 2, 3, 4, 5, 6, 7, 8], 3))
# [[1, 2, 3], [4, 5, 6], [7, 8]]
group_1, group_2 = distribute(2, [1, 2, 3, 4, 5, 6])
list(group_1) # [1, 3, 5]
list(group_2) # [2, 4, 6]
children = divide(3, [1, 2, 3, 4, 5, 6, 7])
@marklit
marklit / places.sql
Last active May 19, 2024 22:43
Pull H3s for Overture's Places Dataset for May 2024
COPY (
WITH a AS (
SELECT h3_cell_to_parent(h3_string_to_h3(SUBSTR(id, 0, 17)), 2) h3_2,
COUNT(*) num_recs
FROM read_parquet('s3://overturemaps-us-west-2/release/2024-05-16-beta.0/theme=places/type=place/*.parquet',
filename=true,
hive_partitioning=1)
GROUP BY 1
)
SELECT h3_cell_to_boundary_wkt(h3_2),
@marklit
marklit / hls.py
Last active June 3, 2023 15:40
Video Stream Generator
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
'''
Please see https://tech.marksblogg.com/streaming-video-hls.html for more details on this script.
MIT License
Copyright (c) 2023 Mark Litwintschik
Permission is hereby granted, free of charge, to any person obtaining a copy
@marklit
marklit / error.md
Last active December 6, 2022 10:57
BigQuery WKB reading error
$ vi test.csv
0101000020777F0000B0BFEC9E3C3C52C0A8CFD556ECA74440
0101000020777F0000982A1895D43D53C0100BB5A6796F4340
0101000020777F000038B4C876BE3B53C0F05A423EE8714340
0101000020777F0000842F4CA60A3E53C058423EE8D96C4340
0101000020777F000094F6065F986C56C0D05D4BC8077D4140
@marklit
marklit / run.py
Created August 26, 2021 19:47
prettymaps CLI
import vsketch
from prettymaps import *
import matplotlib.font_manager as fm
from matplotlib import pyplot as plt
import click
@click.command()
@click.option('--location', default='Old Town, Tallinn')
@click.option('--radius', default=1000, type=int)
@marklit
marklit / cmd.sh
Last active November 16, 2017 23:00
World Map in ANSI
wget https://eol.jsc.nasa.gov/videos/crewearthobservationsvideos/World_map_blank_small.png
convert World_map_blank_small.png World_map_blank_small.jpg
jp2a --colors --term-fit -i World_map_blank_small.jpg | sed -r 's/[XM]/ /g'

Keybase proof

I hereby claim:

  • I am marklit on github.
  • I am marklit (https://keybase.io/marklit) on keybase.
  • I have a public key ASARIRAX9pGFl6YOQvWpSrl-DusmWqCP_cFNr002Z29pEwo

To claim this, I am signing this object: