Skip to content

Instantly share code, notes, and snippets.

View jbants's full-sized avatar

James Banting jbants

View GitHub Profile
@marklit
marklit / places.sql
Last active May 19, 2024 22:43
Pull H3s for Overture's Places Dataset for May 2024
COPY (
WITH a AS (
SELECT h3_cell_to_parent(h3_string_to_h3(SUBSTR(id, 0, 17)), 2) h3_2,
COUNT(*) num_recs
FROM read_parquet('s3://overturemaps-us-west-2/release/2024-05-16-beta.0/theme=places/type=place/*.parquet',
filename=true,
hive_partitioning=1)
GROUP BY 1
)
SELECT h3_cell_to_boundary_wkt(h3_2),
@celoyd
celoyd / despeckling.md
Last active September 1, 2021 14:47
Learning to despeckle sar with speckly targets

Learning to despeckle sar with speckly targets

These are notes from a one-day project to test a hunch. The idea is to train a convolutional neural network to remove speckle from sar (synthetic aperture radar) using only one other observation – with its own speckles – as the target. This method does not come close to state of the art despeckling, and can be biased by the skewed distribution of noise in a way that makes it useless for quantitative research. However, I hadn’t noticed it in the literature and I think it’s kind of funny, so I’m writing it up.

Everything here is about Sentinel-1 L1 GRD-HD data, since it’s what I used, since it’s free.

Speckle

Sar observations contain speckle, a form of interference related to the sparkles in reflected laser light. By some definitions speckle is not noise, since it’s physically real outside the sensor and contains information, but we will treat it as noise. Speckle is (close enough to) independent between radar chirps, a.k.a. looks, and even its distributio

Pansharpening notes, mid-2021

First posted in August 2021. This is basically a snapshot of my thinking about pansharpening at that time; I’m not making any substantial updates. Last typo and clarity fixes in February 2023.

Preface

This is a collection of notes on how I’ve been approaching convolutional neural networks for pansharpening. It’s an edited version of an e-mail to a friend who had asked about this tweet, so it’s informal and somewhat silly; it’s not as polished as, say, a blog post would be. It’s basically the advice I would give to an image processing hobbyist before they started working on pansharpening.

If you want a more serious introduction, start with the literature review in Learning deep multiresolution representations for pansharpening. Most of the academic work I would recommend is mentioned there.

# -*- coding: utf-8 -*-
""" Deletes all tweets below a certain retweet threshold.
"""
import tweepy
from datetime import datetime
# Constants
CONSUMER_KEY = ''

Dockerfile

FROM lambci/lambda:build-python3.6

# Install python module needed for gdal and rasterio
RUN yum update -y && yum clean all
RUN yum install -y yum-plugin-ovl || true
RUN pip3 install cython numpy --no-binary numpy

# Install apt dependencies fro GDAL and drivers

RUN yum install -y libjpeg-devel zlib-devel libpng-devel libcurl-devel \

@philippslang
philippslang / launch.json
Created July 6, 2018 10:28
VS Code pytest debugging settings
{
"version": "0.2.0",
"configurations": [
{
"name": "Python: loadbalancer test",
"type": "python",
"request": "launch",
"module": "pytest",
"args": [
"--no-cov"
@sgreben
sgreben / apk flags.md
Created April 13, 2018 13:09
apk flags

apk

apk-tools 2.8.2, compiled for x86_64.

usage: apk COMMAND [-h|--help] [-p|--root DIR] [-X|--repository REPO] [-q|--quiet] [-v|--verbose] [-i|--interactive] [-V|--version] [-f|--force]
           [--force-binary-stdout] [--force-broken-world] [--force-non-repository] [--force-old-apk] [--force-overwrite] [--force-refresh] [-U|--update-cache]
           [--progress] [--progress-fd FD] [--no-progress] [--purge] [--allow-untrusted] [--wait TIME] [--keys-dir KEYSDIR] [--repositories-file REPOFILE]
           [--no-network] [--no-cache] [--cache-dir CACHEDIR] [--arch ARCH] [--print-arch] [ARGS]...
@sgillies
sgillies / advanced_rasterio_features.ipynb
Last active June 18, 2024 16:03
Advanced Rasterio features notebook
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@suewonjp
suewonjp / Edit and review github wiki on your local machine.md
Last active February 11, 2023 22:35
Edit and review github wiki on your local machine
First, clone your github wiki to your local machine
  1. Checkout your repository wiki via git.
     - You can find the URL at the lower-right of your wiki page (look for the label "Clone this wiki locally")
  2. Now that you have pulled down a local copy of your github repo wiki, create a new file in the repo called _Sidebar.md  - Note that other names might not work.
  3. Within the newly created _Sidebar.md file, you can add appropriate [[link]] markdown syntax.
  4. Add your new file to the local repository, and push via git push origin master.

With the same procedure, you can add a header(_Header.md) and footer(_Footer.md) file.