Skip to content

Instantly share code, notes, and snippets.

@walkermatt
walkermatt / debounce.py
Created June 4, 2012 21:44
A debounce function decorator in Python similar to the one in underscore.js, tested with 2.7
from threading import Timer
def debounce(wait):
""" Decorator that will postpone a functions
execution until after wait seconds
have elapsed since the last time it was invoked. """
def decorator(fn):
def debounced(*args, **kwargs):
def call_it():
@walkermatt
walkermatt / mapproxy_decorate.py
Last active March 22, 2024 16:48
Minimal MapProxy Middleware demonstrating the decorate_img API
""" Minimal MapProxy Middleware demonstrating the decorate_img API
To run:
1. Install MapProxy in a virtual enviroment together with Gunicorn
2. Create a basic MapProxy config and copy this file into the same directory as mapproxy.yaml
2. Activate virtual environment
3. Change to the directory containing this file
4. Run:
gunicorn -k eventlet --workers=1 --log-file=- mapproxy_decorate:application
@walkermatt
walkermatt / main.js
Created February 2, 2024 10:21
Parse a directory of nginx access logs to get a list of referer values
import { Parser } from '@robojones/nginx-log-parser';
import fs from 'fs';
import { createReadStream } from 'fs';
import zlib from 'zlib';
import readline from 'readline';
import path from 'path';
async function* processFile(filePath, filter) {
const extension = path.extname(filePath);
let readStream;
@walkermatt
walkermatt / README.md
Last active October 30, 2023 13:57
Chalice Dev AWS IAM Policy

Chalice Dev AWS IAM Policy

AWS IAM Policy suitable for assigning to a user developing Chalice applications. Allows the developer sucessfully execute chalice deploy, chalice delete and chalice logs

Derived from comments on chalice/issues/59, extended by trial and error :-)

Subsitute YOUR-AWS-REGION in chalice-dev-iam-policy.json with the region you are deploying to, for example eu-west-1.

AWS profiles

@walkermatt
walkermatt / ogr_gfs_geom_types.md
Last active October 10, 2023 08:31
List of geometry type codes that can specified in an ogr gfs file.

OGR GFS Geometry Values

The geometry type of a feature class can be specified in a .gfs file used by OGR to map from a GML document to a simple feature schema. The following numbers can be used to specify a specific geometry type:

-2147483647 Point25D
-2147483646 LineString25D
-2147483645 Polygon25D
-2147483644 MultiPoint25D
-2147483643 MultiLineString25D

-2147483642 MultiPolygon25D

@walkermatt
walkermatt / wfs2postgis.sh
Created September 26, 2012 20:09
WFS to PostGIS
# Example of downloading all data from a GeoServer WFS server
# and loading it into a PostGIS database. Use with caution
# as you could put a lot of load on someone's server if they
# host a lot of data which might make them sad.
# In response to: http://underdark.wordpress.com/2012/09/26/wfs-to-postgis-in-3-steps/
BASEURL="http://data.wien.gv.at/daten/geoserver/ows?service=WFS&version=1.1.0"
for LAYERNAME in `wget -qO- $BASEURL"&request=GetCapabilities" | xpath -q -e "//FeatureType/Name/text()"` ; do
PARTS=(${LAYERNAME//:/ })
@walkermatt
walkermatt / foss4guk2023_foss4g2013_ten_years_after.md
Last active September 12, 2023 09:24
Note from FOSS4G UK 2023 "Ten Years After" talk

FOSS4G 2013 - Ten Years After

Matt Walker mattwalker@astuntechnology.com, x.com/_walkermatt

Data

Improved availability and visibility

  • Open data
    • OSM founded almost 10 years prior to FOSS4G 2013
  • Wide adoption; Humanitarian OSM (hotosm.org)
@walkermatt
walkermatt / testutil.sql
Last active July 30, 2023 15:17
Postgres testing utilities
-- Raises an exception with `failure_message` if `sql_text` does not throw an exception at all.
-- If an exception is thrown but it's SQLSTATE doesn't match `expected_sql_state` then that
-- exception is RE-RAISED
CREATE OR REPLACE FUNCTION pg_temp.test__throws_exception(sql_text text, expected_sql_state text, failure_message text) RETURNS void AS $$
DECLARE
sql_state text;
BEGIN
EXECUTE sql_text;
RAISE EXCEPTION '%', failure_message;
EXCEPTION WHEN OTHERS THEN GET STACKED DIAGNOSTICS sql_state = RETURNED_SQLSTATE;
@walkermatt
walkermatt / bbox-to-wkt-polygon.js
Created August 16, 2015 15:04
Create WKT POLYGON from a bounding box in JavaScript
var str = "POLYGON((left top,right top,right bottom,left bottom,left top))";
var bbox = {left: 10, top: 30, right: 20, bottom: 40};
str.replace(/[a-z]+/g, function(s) {return bbox[s];});
@walkermatt
walkermatt / README.md
Last active May 9, 2023 15:44
Example of requesting JSON data from a URL and converting to CSV

Download a JSON list of object and write to CSV

Requires Python 3.

Usage

Linux

python3 json_to_csv.py https://gist.githubusercontent.com/walkermatt/19b1108ca9bee94745810f4c39c32165/raw/cities.json /tmp/cities.csv