Skip to content

Instantly share code, notes, and snippets.

View walkermatt's full-sized avatar
💭
Drinking tea

Matt Walker walkermatt

💭
Drinking tea
View GitHub Profile
@walkermatt
walkermatt / README.md
Last active October 30, 2023 13:57
Chalice Dev AWS IAM Policy

Chalice Dev AWS IAM Policy

AWS IAM Policy suitable for assigning to a user developing Chalice applications. Allows the developer sucessfully execute chalice deploy, chalice delete and chalice logs

Derived from comments on chalice/issues/59, extended by trial and error :-)

Subsitute YOUR-AWS-REGION in chalice-dev-iam-policy.json with the region you are deploying to, for example eu-west-1.

AWS profiles

@walkermatt
walkermatt / README.md
Last active May 9, 2023 15:44
Example of requesting JSON data from a URL and converting to CSV

Download a JSON list of object and write to CSV

Requires Python 3.

Usage

Linux

python3 json_to_csv.py https://gist.githubusercontent.com/walkermatt/19b1108ca9bee94745810f4c39c32165/raw/cities.json /tmp/cities.csv
@walkermatt
walkermatt / testutil.sql
Last active July 30, 2023 15:17
Postgres testing utilities
-- Raises an exception with `failure_message` if `sql_text` does not throw an exception at all.
-- If an exception is thrown but it's SQLSTATE doesn't match `expected_sql_state` then that
-- exception is RE-RAISED
CREATE OR REPLACE FUNCTION pg_temp.test__throws_exception(sql_text text, expected_sql_state text, failure_message text) RETURNS void AS $$
DECLARE
sql_state text;
BEGIN
EXECUTE sql_text;
RAISE EXCEPTION '%', failure_message;
EXCEPTION WHEN OTHERS THEN GET STACKED DIAGNOSTICS sql_state = RETURNED_SQLSTATE;
@walkermatt
walkermatt / foss4guk2023_foss4g2013_ten_years_after.md
Last active September 12, 2023 09:24
Note from FOSS4G UK 2023 "Ten Years After" talk

FOSS4G 2013 - Ten Years After

Matt Walker mattwalker@astuntechnology.com, x.com/_walkermatt

Data

Improved availability and visibility

  • Open data
    • OSM founded almost 10 years prior to FOSS4G 2013
  • Wide adoption; Humanitarian OSM (hotosm.org)
@walkermatt
walkermatt / main.js
Created February 2, 2024 10:21
Parse a directory of nginx access logs to get a list of referer values
import { Parser } from '@robojones/nginx-log-parser';
import fs from 'fs';
import { createReadStream } from 'fs';
import zlib from 'zlib';
import readline from 'readline';
import path from 'path';
async function* processFile(filePath, filter) {
const extension = path.extname(filePath);
let readStream;