celoyd

Pansharpening notes, mid-2021

First posted in August 2021. This is basically a snapshot of my thinking about pansharpening at that time; I’m not making any substantial updates. Last typo and clarity fixes in February 2023.

Preface

This is a collection of notes on how I’ve been approaching convolutional neural networks for pansharpening. It’s an edited version of an e-mail to a friend who had asked about this tweet, so it’s informal and somewhat silly; it’s not as polished as, say, a blog post would be. It’s basically the advice I would give to an image processing hobbyist before they started working on pansharpening.

If you want a more serious introduction, start with the literature review in Learning deep multiresolution representations for pansharpening. Most of the academic work I would recommend is mentioned there.

celoyd

Getting GSOD

0. Introduction

0₀: What’s in this guide?

This is a discursive recipe for turning the Global Summary of the Day weather dataset into a useful PostgreSQL database on a Unix system. It's aimed at data nerds with no prior knowledge of GSOD. For simplicity, it only describes one way of doing things, but the canny reader will see many possible variations (for example, using staged temporary files rather than a pipeline to do data conversion) and substitutions (mysql for postgres, curl for wget, perl for python, …).

Depending on how you do things, you’ll need something like 50 free gigabytes of disk space to work in, and the final database will use about 20 gigabytes.

celoyd

Notes on a big SAR swath

Introduction

I just had a big mug of coffee and now I have some black tea and here is a slapdash devlog about a “A lightly annotated SAR view from Los Angeles to Woss”.

Summary

This is basically a combination of several conversations I’ve had with friends about different aspects of the project.

celoyd

This is how to make animations like this (alternate), inspired by Paul Norman’s. This is a write-up of a one-off 45 minute project, so it’s rough around the edges and probably has a few typos; feel free to point them out. It’s mostly command-line work, using tools like GNU parallel and ImageMagick convert; it’s slow and wastes a lot of filesystem space compared to a more monolithic approach in (say) python, but it’s very flexible.

1. Get data

I use curl globs, for example:

mkdir xzs
cd xzs
curl -O 'https://planet.openstreetmap.org/tile_logs/tiles-2022-[01-12]-[01-31].txt.xz'

celoyd

"""
draw_osm_day.py osm_tile_traffic_file.txt output_image.tiff

Write a float32 TIFF representing tile traffic in the OSM aggregate log.

Charlie Loyd, 2023-09-14. Inspired by https://en.osm.town/@pnorman/111062780453196500.

I recommend downloading the .xz compressed files and using xzcat to feed
them to this script. (Doing xz decompression in-script was slow.) E.g.,

celoyd

import requests as req
import sys
from dateutil.parser import parse
from PIL import Image
from StringIO import StringIO

# hi8-fetch.py <date> <zoom level> <output>
# E.g.: hi8-fetch.py 2016-01-13T22:10:00 8 2016-01-13T221000-z8.png
# Fetch Himawari-8 full disks at a given zoom level.
# Valid zoom levels seem to be powers of 2, 1..16, and 20.

celoyd

Dear Ms Liang,

I am an Oakland resident and a past and future user of OAK.

I want a plan to make OAK a source of local pride and global leadership in responding to the climate crisis. That is not the plan on the table today. This plan only gestures at the climate crisis and aviation’s role in it, and invests nothing serious to help solve it. In fact, what’s proposed would make OAK an increasing part of the problem for years to come.

The plan is insufficient to our moment. I oppose it firmly enough to invest my personal time and resources to publicly work against it.

I hope you will reconsider and submit a better plan. I would expect it to start from the principle that any change to OAK must make it responsible for less CO2e emission. That is, the net climate impact of the airport, calculated globally and not just to the mixing level, and operating as designed, must be at worst steady.

celoyd

import numpy as np
from skimage import io
import time
import datetime
from ciso8601 import parse_datetime
from sys import argv

timestep = 60  # in seconds
lonsteps = 1  # pixels per degree, i.e., 1 means an image 360 wide, 2 means 720

celoyd

#!/usr/bin/env python

# simple visualization of the Cb/Cr plane of an image
# python vectorscope.py input.png output.png
# (supports whatever image formats your skimage does)
# bugs include: everything is in painter order

from skimage import io
import numpy as np
from sys import argv

celoyd

#!/usr/bin/env python

# ndvi.py red.tif nir.tif output-ndvi.tif
# Calculate NDVI (see Wikipedia). Assumes atmospheric correction.
# (Although I use it without all the time for quick experiments.)

import numpy as np
from sys import argv
from osgeo import gdal, gdalconst