Skip to content

Instantly share code, notes, and snippets.

Matthew Perry perrygeo

Block or report user

Report or block perrygeo

Hide content and notifications from this user.

Learn more about blocking users

Contact Support about this user’s behavior.

Learn more about reporting abuse

Report abuse
View GitHub Profile
View pgextwlist.md

Installing pgextwlist

# Dockerfile
ENV PGEXTWLIST_VERSION 1.7
RUN wget -q -O pgextwlist-${PGEXTWLIST_VERSION}.tar.gz https://github.com/dimitri/pgextwlist/archive/v${PGEXTWLIST_VERSION}.tar.gz
RUN tar -xzf pgextwlist-${PGEXTWLIST_VERSION}.tar.gz && \
    cd pgextwlist-${PGEXTWLIST_VERSION} && \
    make -j${CPUS} && make install && \
    export pkg=$(pg_config --pkglibdir) && mkdir $pkg/plugins && cp $pkg/pgextwlist.so $pkg/plugins
@perrygeo
perrygeo / output_cHOUF3.gif
Last active Sep 7, 2018
transform_geom warping bug, sensitivity to central meridian
@perrygeo
perrygeo / tiled_assert.py
Last active Jun 14, 2018
Tiled=true fails silently
View tiled_assert.py
#!/usr/bin/env python
from pprint import pprint
from functools import reduce
import operator
import numpy as np
from affine import Affine
from rasterio.crs import CRS
import rasterio
View send_message.py
import random
import time
import json
import threading
import click
def send_message(msg, async=True):
"""Logs the message to an output stream.
@perrygeo
perrygeo / log_calls.py
Created Mar 20, 2018
Decorator to log all call parameters and return values for a function
View log_calls.py
import sys
def log_calls(fh=sys.stderr):
"""Logs the function, args, kwargs and return value to a file handle
"""
def decorator(func):
def wrapper(*args, **kwargs):
retval = func(*args, **kwargs)
print(func, args, kwargs, retval, file=fh)
@perrygeo
perrygeo / Dockerfile
Last active Oct 2, 2019
Minimal debian image with Python 3.6 and geo python tools
View Dockerfile
FROM python:3.6-slim-stretch
ADD requirements.txt /tmp/requirements.txt
RUN apt-get update && \
apt-get install -y \
build-essential \
make \
gcc \
locales \
@perrygeo
perrygeo / deprecated_kwarg.py
Last active Jan 9, 2018
Decorator for aliasing kwargs in Python functions
View deprecated_kwarg.py
#!/usr/bin/env python
import warnings
from functools import wraps
def deprecated_kwarg(old, new, version):
def decorator(func):
@wraps(func)
def func_wrapper(*args, **kwargs):
new_kwargs = kwargs.copy()
View plot_fit_ride.py
#!/usr/bin/env python
from fitparse import FitFile
import click
import pandas as pd
import matplotlib.pyplot as plt
from matplotlib.ticker import MaxNLocator
def get_records(fitfile, fields=None):
if fields is None:
View parallel_concurrent_python.md

Parallelism and Concurrency in Python

There should be one-- and preferably only one --obvious way to do it.

-- Tim Peters, Zen of Python

When we need our programs to run faster, the first place we often look is parallelism and concurrency. By doing more things at once, we hope for significant speed gains. Python has a number of techniques optimized for different use cases; there is no one obvious way to do it! So how do you decide the correct approach?

Here's my quick take on the primary tools for parallel execution in Python, their strengths and weaknesses. I'll focus mainly on the tools in the standard library, but I'll touch on a few third-party modules that provide interesting features.

View Makefile.Rasterio
all: deps clean install test
.PHONY: docs
install:
python setup.py build_ext
pip install -e .[all]
deps:
pip install -r requirements-dev.txt
You can’t perform that action at this time.