celoyd

Introduction

I was writing a note to a friend that mentioned my tedious opinions on “AI” discourse. It veered off into my usual argument that big “AI” companies are shaping the industry ecosystem to their own ends by setting up a situation where expensive-to-run models are overvalued. I think they’re doing this because they have a competitive advantage in that tier of the market, having bought (time on) a lot of GPUs. It’s like how a company that owns diamond mines will probably promote the idea that large, mined diamonds are important and valuable, and that there’s something off about running a sub-industrial mine or lab-growing diamonds. You can do this without lying at all, but I still dislike it. Large mined diamonds here are $O(n^2)$ models.

To support this argument, I started making my case against the necessity of the standard transformer model. I admit that the case is scattershot and circumstantial. It’s not that SDPA (the normal transformer architecture) is a fraud, or that there is something m

celoyd

Pansharpening notes, mid-2021

First posted in August 2021. This is basically a snapshot of my thinking about pansharpening at that time; I’m not making any substantial updates. Last typo and clarity fixes in February 2023. December 2025 update: many ideas here, and some newer ones, are demonstrated in Potato.

Preface

This is a collection of notes on how I’ve been approaching convolutional neural networks for pansharpening. It’s an edited version of an e-mail to a friend who had asked about this tweet, so it’s informal and somewhat silly; it’s not as polished as, say, a blog post would be. It’s basically the advice I would give to an image processing hobbyist before they started working on pansharpening.

If you want a more serious introduction, start with the literature review in [Learning deep multiresolution representations for pansharpening](https://ar

celoyd

# nd.py a.tiff b.tiff difference.tiff
# quick and dirty (opinionated and not robust)
# assumes same-size single-band inputs
# output always uint16
# error always 2**15

from sys import argv
import rasterio as rio
from pathlib import Path
import numpy as np

celoyd

#!/usr/bin/env python

"""Simple slitscan generator

$ slitscan.py source slice destination

Where:
- source is a video or a directory of images;
- slice is either "c" for column or "r" for row
  and an integer, e.g., r1 or c42; and

celoyd

# python neffer.py 55 111879
#          mission ^^ ^^^^^^ frame
# will write to iss055e111878.nef.

# This code is public domain; please improve & adapt it.

import requests
from sys import argv, exit
from random import uniform
from time import sleep

celoyd

from skimage import io
from pathlib import Path
from sys import argv

srtm_dir = Path(argv[1])

width = 3601  # pixel width of one of these srtm tiffs

starting_lon = -2
ending_lon = 143

celoyd

Getting GSOD

0. Introduction

0₀: What’s in this guide?

This is a discursive recipe for turning the Global Summary of the Day weather dataset into a useful PostgreSQL database on a Unix system. It's aimed at data nerds with no prior knowledge of GSOD. For simplicity, it only describes one way of doing things, but the canny reader will see many possible variations (for example, using staged temporary files rather than a pipeline to do data conversion) and substitutions (mysql for postgres, curl for wget, perl for python, …).

Depending on how you do things, you’ll need something like 50 free gigabytes of disk space to work in, and the final database will use about 20 gigabytes.

celoyd

Notes on a big SAR swath

Introduction

I just had a big mug of coffee and now I have some black tea and here is a slapdash devlog about a “A lightly annotated SAR view from Los Angeles to Woss”.

Summary

This is basically a combination of several conversations I’ve had with friends about different aspects of the project.

celoyd

This is how to make animations like this (alternate), inspired by Paul Norman’s. This is a write-up of a one-off 45 minute project, so it’s rough around the edges and probably has a few typos; feel free to point them out. It’s mostly command-line work, using tools like GNU parallel and ImageMagick convert; it’s slow and wastes a lot of filesystem space compared to a more monolithic approach in (say) python, but it’s very flexible.

1. Get data

I use curl globs, for example:

mkdir xzs
cd xzs
curl -O 'https://planet.openstreetmap.org/tile_logs/tiles-2022-[01-12]-[01-31].txt.xz'

celoyd

"""
draw_osm_day.py osm_tile_traffic_file.txt output_image.tiff

Write a float32 TIFF representing tile traffic in the OSM aggregate log.

Charlie Loyd, 2023-09-14. Inspired by https://en.osm.town/@pnorman/111062780453196500.

I recommend downloading the .xz compressed files and using xzcat to feed
them to this script. (Doing xz decompression in-script was slow.) E.g.,