Skip to content

Instantly share code, notes, and snippets.

@perrygeo
perrygeo / base64_padding.md
Last active October 25, 2023 16:20
Avoiding TypeError: Incorrect padding with Python's base64 encoding

Avoiding padding errors with Python's base64 encoding

>>> import base64
>>> data = '{"u": "test"}'
>>> code = base64.b64encode(data)
>>> code
'eyJ1IjogInRlc3QifQ=='
@perrygeo
perrygeo / _partial_pixel_rasterization.md
Last active October 12, 2023 17:10
partial pixel rasterization

This is a proof-of-concept for a numpy/rasterio/shapely based implementation of partial coverage rasterization. It works, barely.

The current GDAL algorithms allow for two methods, both binary: the default centroid method and the all-touched method.

This is a third alternative which provides the percentage of coverage of each cell from 0 to 100 which can be thought of as pixel weights for many spatial analyses.

See discussion at rasterio/rasterio#232

@perrygeo
perrygeo / loading spatialite data into GeoDataFrames.md
Last active October 1, 2023 16:57
Loading spatialite tables into geopandas

Loading spatialite tables into GeoPandas GeoDataFrames

This little trick derives from the fact that the from_postgis class method is not really specific to postgis at all; it will work with sqlalchmey or dbapi2 connections. However, there are some peculiarities with spatialite that prevent this from being as simple as one might hope. There are two options:

  • The pysqlite2 driver works great for vanilla sqlite3 databases but spatialite requires loading an extension. In order to load extensions, you need to install a patched version and do some manual loading of the shared library. (see the_pysqlite2_way.py)

  • A better alternative is to use pyspatialite but installation is also a bit funky. I had to install from the current git master instead of the pypi version. The extension is loaded automatically. (see the_pyspatialite_way.py)

@perrygeo
perrygeo / django_model_graph.sh
Created April 13, 2013 21:44
Generate UML diagram of django app models
apt-get install python-pygraphviz
pip install django-extensions
# add 'django_extensions' to INSTALLED_APPS in settings.py
python manage.py graph_models trees -o test.png
@perrygeo
perrygeo / osm-tiledb.txt
Created June 23, 2023 12:05
Convert Open Street Map data into TileDB arrays
# Using osm.pbf extracts from https://download.geofabrik.de/index.html
# Requires GDAL 3.7+ with the TileDB driver
ogr2ogr -f TileDB ./dc-lines district-of-columbia-latest.osm.pbf lines
ogr2ogr -f TileDB ./dc-points district-of-columbia-latest.osm.pbf points
ogr2ogr -f TileDB ./dc-polygons district-of-columbia-latest.osm.pbf multipolygons
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@perrygeo
perrygeo / zonal_stats.py
Last active March 22, 2023 05:01
Python implementation of zonal statistics function. Optimized for dense polygon layers, uses numpy, GDAL and OGR to rival the speed of starspan.
"""
Zonal Statistics
Vector-Raster Analysis
Copyright 2013 Matthew Perry
Usage:
zonal_stats.py VECTOR RASTER
zonal_stats.py -h | --help
zonal_stats.py --version
@perrygeo
perrygeo / Dockerfile
Last active February 27, 2023 12:00
Minimal debian image with Python 3.6 and geo python tools
FROM python:3.6-slim-stretch
ADD requirements.txt /tmp/requirements.txt
RUN apt-get update && \
apt-get install -y \
build-essential \
make \
gcc \
locales \
@perrygeo
perrygeo / mercator_scale.py
Last active February 17, 2023 08:10
Web Mercator Scale/Resolution Calculations
"""
Web Mercator Scale and Resolution Calculations
Python implementation of the formulas at http://msdn.microsoft.com/en-us/library/bb259689.aspx
"""
import math
def meters_per_pixel(zoom, lat):
"""
ground resolution = cos(latitude * pi/180) * earth circumference / map width
"""
DROP TABLE if EXISTS boundaries;
CREATE TABLE boundaries (
id BIGSERIAL primary key,
geometry geometry(multipolygon, 4326) NOT null
);
CREATE INDEX idx_boundaries_geometry
ON boundaries
USING gist (geometry);