Skip to content

Instantly share code, notes, and snippets.

View kgjenkins's full-sized avatar

Keith Jenkins kgjenkins

View GitHub Profile
@kgjenkins
kgjenkins / hash.py
Created May 3, 2024 17:45
generate hashes for a csv of filepaths
# this script goes through a list of files from Excel "data from folder"
# and computes a hash of the file contents (to be used for deduplication)
import csv
import os
import hashlib
import time
# columns: Name,Extension,Date accessed,Date modified,Date created,Folder Path,Hash
@kgjenkins
kgjenkins / readme.md
Last active May 3, 2024 12:21
Creating halftones in QGIS

Creating halftones in QGIS

A quick demo using a hexagon grid with a centroid fill of points sized using raster values from the image layer.

qgis-hex-halftones2

@kgjenkins
kgjenkins / README.md
Created April 19, 2024 16:45
variation across multiple scans

The same image (an illustrated book cover) was scanned on the flatbed scanner (EPSON WF-2630) three times at 300dpi.

Analysis of the variation across the three scans:

band min max mean std dev
red 0 36 4.47 2.59
green 0 36 4.41 2.68
blue 0 37 4.72 2.95
@kgjenkins
kgjenkins / index.py
Created April 17, 2024 20:45
Calculate which city points are within each tif in a directory
# this script goes through a directory of tif files and a shapefile of cities
# and reports which city points are contained within each tif
import os
import fiona
import rasterio
import shapely
from pyproj import Transformer
tifpath = "path/to/tiffs"
@kgjenkins
kgjenkins / gist:ad26a278ecbf745873b8504dd7e217b3
Last active March 1, 2024 21:02
Calculating a raster of the network distance to the nearest park
The limiting factor in terms of the speed of the Iso-areas tool seems
to be the number of starting points. I've tried a number of
variations, trying to reduce the number of points while retaining
enough points that the analysis is still accurate. After several
failed attempts, I think I've come up with a workflow that ends up
keeping just the points along the park perimeters that end up being
the closest points to the road network. Here's the general workflow,
assuming we are starting with 2 layers, "streets" and "parks". I'd
suggest creating temporary outputs for each of these steps, and then
make the final result permanent.
@kgjenkins
kgjenkins / readme.md
Last active January 27, 2024 22:01
TIFF compression options

TIFF compression options

Summary

Realistically, especially when considering the inherent noise in the original image, I'd settle for lossy compression with COMPRESS=JPEG JPEG_QUALITY=90, which could reduce file size fairly quickly to 16% of the original.

But if lossless compression is a hard requirement, I'd probably go with COMPRESS=LZW PREDICTOR=2. However, I would want to verify that any downstream tools would support this sort of compression.

UPDATE: When saving with JPEG compression, further speed and size improvements are gained by adding PHOTOMETRIC=YCBCR, which uses a different color space that has even better compression. I've added new rows to the table for YCBCR, as well as .jp2 formats.

@kgjenkins
kgjenkins / index.html
Last active December 4, 2023 06:22
Cornell Theses and Dissertations by Academic Discipline
<html>
<head>
<title>Cornell Theses and Dissertations by Academic Discipline</title>
<meta http-equiv='Content-type' content='text/html; charset=UTF-8'>
<meta name='author' content='Keith Jenkins'>
<meta name='dateModified' content='2022-01-11'>
<style>
body { font-family: sans-serif }
h1 { text-align:center ; background: #b31b1b ; color: #fff ; padding: 0.5em ; margin: 0 }
td { white-space: nowrap }
@kgjenkins
kgjenkins / README.md
Last active September 12, 2023 20:45
Reclassifying UrbanWatch rasters using QGIS and Rasterio

Reclassifying UrbanWatch rasters using QGIS and Rasterio

See the video demo at https://youtu.be/t-pb5-1Q_Qg

The files here were used to convert UrbanWatch from RGB images to numeric rasters with simple landcover values 0-9, as follows:

Value Category Color
0 No Data Black
1 Building Red
@kgjenkins
kgjenkins / README.md
Last active September 12, 2023 20:32
Reclassifying UrbanWatch rasters using ArcGIS

Reclassifying UrbanWatch rasters using ArcGIS

See the video demo at https://youtu.be/mdYnBFYqXpg

The files here can be used to convert UrbanWatch rasters from RGB images to numeric rasters with simple landcover values 0-9, as follows:

Value Category Color
0 No Data Black
1 Building Red
@kgjenkins
kgjenkins / readme.md
Last active February 21, 2023 18:53
Interpolate points along lines using QGIS virtual layers

Interpolate points along lines using QGIS virtual layers

Suppose we want to interpolate a number of points along each line in a layer:

image

We have a column called stations that specifies how many points to interpolate on each line:

image