Skip to content

Instantly share code, notes, and snippets.

View sgillies's full-sized avatar

Sean Gillies sgillies

View GitHub Profile
@sgillies
sgillies / stats.py
Last active April 26, 2023 08:38
Simple and fancy ways of computing stats of a rasterio dataset
"""Three ways of computing stats of a rasterio dataset."""
import dask.array
import numpy
import rasterio
from rasterio.windows import Window
# Compute mean of entire dataset read into memory. Easiest, but doesn't
# scale for very large datasets.
with rasterio.open(
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@sgillies
sgillies / cfrw.py
Last active April 26, 2023 08:37
Rasterio concurrency example
"""Concurrent read-process-write example"""
import concurrent.futures
from itertools import islice
from time import sleep
import rasterio
CHUNK = 100
@sgillies
sgillies / advanced_rasterio_features.ipynb
Last active November 20, 2023 12:50
Advanced Rasterio features notebook
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
{"geometry": {"coordinates": [[[-111.73527526855469, 41.995094299316406], [-111.65931701660156, 41.99627685546875], [-111.6587142944336, 41.9921875], [-111.65888977050781, 41.95676803588867], [-111.67082977294922, 41.91230010986328], [-111.67332458496094, 41.905494689941406], [-111.67088317871094, 41.90049362182617], [-111.66474914550781, 41.893211364746094], [-111.6506576538086, 41.875465393066406], [-111.64759826660156, 41.87091827392578], [-111.64640808105469, 41.86273956298828], [-111.64334869384766, 41.858192443847656], [-111.63720703125, 41.85499572753906], [-111.633544921875, 41.847267150878906], [-111.63053894042969, 41.83409118652344], [-111.6330337524414, 41.82728576660156], [-111.63983154296875, 41.8227653503418], [-111.6484603881836, 41.82188034057617], [-111.66077423095703, 41.82327651977539], [-111.6712417602539, 41.82330322265625], [-111.67618560791016, 41.82013702392578], [-111.68803405761719, 41.78792953491211], [-111.69361114501953, 41.77931594848633], [-111.70162200927734, 41.77797317504883
@sgillies
sgillies / latency.txt
Created October 19, 2016 11:54 — forked from jboner/latency.txt
Latency Numbers Every Programmer Should Know
Latency Comparison Numbers
--------------------------
L1 cache reference 0.5 ns
Branch mispredict 5 ns
L2 cache reference 7 ns 14x L1 cache
Mutex lock/unlock 25 ns
Main memory reference 100 ns 20x L2 cache, 200x L1 cache
Compress 1K bytes with Zippy 3,000 ns 3 us
Send 1K bytes over 1 Gbps network 10,000 ns 10 us
Read 4K randomly from SSD* 150,000 ns 150 us ~1GB/sec SSD
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@sgillies
sgillies / benchmark.py
Last active February 7, 2019 01:02
GeoJSON: json vs msgpack vs shapely.wkb vs geobuf
# Comparing serialization of complex GeoJSON geometries using:
#
# - standard lib json, marshal, pickle, cPickle
# - umsgpack
# - shapely.wkb
# - geobuf (protobuf)
#
# The test case is a nearly circular polygon with 128 vertices.
#
# Python 2.7 because geobuf isn't possible on Python 3 (because
@sgillies
sgillies / issue253.ipynb
Created April 27, 2015 15:53
Shapely issue 253
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.