Skip to content

Instantly share code, notes, and snippets.

@walkermatt
walkermatt / main.js
Created February 2, 2024 10:21
Parse a directory of nginx access logs to get a list of referer values
import { Parser } from '@robojones/nginx-log-parser';
import fs from 'fs';
import { createReadStream } from 'fs';
import zlib from 'zlib';
import readline from 'readline';
import path from 'path';
async function* processFile(filePath, filter) {
const extension = path.extname(filePath);
let readStream;
@walkermatt
walkermatt / foss4guk2023_foss4g2013_ten_years_after.md
Last active September 12, 2023 09:24
Note from FOSS4G UK 2023 "Ten Years After" talk

FOSS4G 2013 - Ten Years After

Matt Walker mattwalker@astuntechnology.com, x.com/_walkermatt

Data

Improved availability and visibility

  • Open data
    • OSM founded almost 10 years prior to FOSS4G 2013
  • Wide adoption; Humanitarian OSM (hotosm.org)
@walkermatt
walkermatt / testutil.sql
Last active July 30, 2023 15:17
Postgres testing utilities
-- Raises an exception with `failure_message` if `sql_text` does not throw an exception at all.
-- If an exception is thrown but it's SQLSTATE doesn't match `expected_sql_state` then that
-- exception is RE-RAISED
CREATE OR REPLACE FUNCTION pg_temp.test__throws_exception(sql_text text, expected_sql_state text, failure_message text) RETURNS void AS $$
DECLARE
sql_state text;
BEGIN
EXECUTE sql_text;
RAISE EXCEPTION '%', failure_message;
EXCEPTION WHEN OTHERS THEN GET STACKED DIAGNOSTICS sql_state = RETURNED_SQLSTATE;
@walkermatt
walkermatt / README.md
Last active May 9, 2023 15:44
Example of requesting JSON data from a URL and converting to CSV

Download a JSON list of object and write to CSV

Requires Python 3.

Usage

Linux

python3 json_to_csv.py https://gist.githubusercontent.com/walkermatt/19b1108ca9bee94745810f4c39c32165/raw/cities.json /tmp/cities.csv
@walkermatt
walkermatt / README.md
Last active October 30, 2023 13:57
Chalice Dev AWS IAM Policy

Chalice Dev AWS IAM Policy

AWS IAM Policy suitable for assigning to a user developing Chalice applications. Allows the developer sucessfully execute chalice deploy, chalice delete and chalice logs

Derived from comments on chalice/issues/59, extended by trial and error :-)

Subsitute YOUR-AWS-REGION in chalice-dev-iam-policy.json with the region you are deploying to, for example eu-west-1.

AWS profiles

@walkermatt
walkermatt / _talk-builds-tests-bugs-semver.md
Last active January 13, 2021 12:36
Talk on small contributions to open source software, testing, automated builds etc.

The day the build broke 😔

The tale of a broken build and breaking changes...


And so the story begins...

  • ol-ishare is built automatically each time a change is committed to the repository
    • Formatting of source code is checked (linting)
@walkermatt
walkermatt / build.sh
Last active November 11, 2020 09:37
ol-layerswitcher migration to TypeScript presentation
npx @marp-team/marp-cli --template bare -o ol-layerswitcher-typescript.html presentation.md
@walkermatt
walkermatt / confluence-table-to-csv.js
Last active February 21, 2018 11:42
Copy the data out of a Confluence table as CSV using Chrome devtools
function escapeAsCsv(str) {
// Double up double quotes and quote the entire string if necessary
if (str.includes(',') || str.includes('\n') || str.includes('"')) {
return '"' + str.replace(/"/g, '""') + '"';
}
return str;
}
// Export Confluence table as CSV
// Select the tbody element in Chrome devtools Elements panel then run the
This file has been truncated, but you can view the full file.
// OpenLayers. See https://openlayers.org/
// License: https://raw.githubusercontent.com/openlayers/openlayers/master/LICENSE.md
// Version: v4.5.0-70-gfca0b07
;(function (root, factory) {
if (typeof exports === "object") {
module.exports = factory();
} else if (typeof define === "function" && define.amd) {
define([], factory);
} else {
root.ol = factory();