Skip to content

Instantly share code, notes, and snippets.

View purcell's full-sized avatar

Steve Purcell purcell

View GitHub Profile
@purcell
purcell / access-log-to-parquet.sql
Created September 4, 2023 19:23
Use DuckDB to convert a compressed web access log in Combined Log Format to Parquet
COPY (
WITH
-- Read the raw log line by line by abusing CSV parser
raw_log AS (
FROM read_csv_auto('/tmp/log/access.log-20230904.gz', header=false, delim='\0')
)
, combined_log AS (
SELECT regexp_extract(column0
, '^(\S+) (\S+) (\S+) \[(.*?)\] "([A-Z]+?) (.*?) HTTP/(.*?)" (\d+) (\d+) "(.*?)" "(.*?)"$'
, [ 'ip', 'identity', 'userid', 'timestamp', 'method'
@purcell
purcell / wrap-pg-auth
Created August 17, 2023 14:23
Invoke a postgres command via a bastion host with auth obtained from 1password
#!/bin/bash -e
error_exit() {
echo "$@" >&2
exit 1
}
usage_exit() {
cat <<EOF >&2
Allows you to run a local postgres command, e.g. psql or pg_dump,
@purcell
purcell / wrap-aws-auth
Created August 17, 2023 14:16
Invoke a command with AWS auth obtained from 1password
#!/bin/bash -e
error_exit() {
echo "$1" >&2
exit 1
}
usage_exit() {
cat <<EOF >&2
Looks up your AWS keys from your 1Password and exports them before running the next command
@purcell
purcell / icfpcs.md
Created July 10, 2023 18:46
My ICFP Contest history
This file has been truncated, but you can view the full file.
@nix { "action": "setPhase", "phase": "unpackPhase" }
unpacking sources
unpacking source archive /nix/store/8lsf1rrc5d3cgn2i1as76l2k3yr16k9r-ocaml-5.0.0.tar.xz
source root is ocaml-5.0.0
setting SOURCE_DATE_EPOCH to timestamp 1671176928 of file ocaml-5.0.0/yacc/wstr.c
@nix { "action": "setPhase", "phase": "patchPhase" }
patching sources
@nix { "action": "setPhase", "phase": "updateAutotoolsGnuConfigScriptsPhase" }
updateAutotoolsGnuConfigScriptsPhase
Updating Autotools / GNU config script to a newer upstream version: ./build-aux/config.sub
@purcell
purcell / keybase.md
Created June 23, 2023 12:37
keybase

Keybase proof

I hereby claim:

  • I am purcell on github.
  • I am sanityinc (https://keybase.io/sanityinc) on keybase.
  • I have a public key ASD6Zb31etK_0lSccFnXOXQKEk2S1137StrRR5dfWAn4Jgo

To claim this, I am signing this object:

@purcell
purcell / 0001-Use-upstreamed-poetry2nix-overrides-for-hid-and-pyus.patch
Created May 11, 2021 03:12
Use upstreamed poetry2nix overrides for hid and pyusb
From 96f7801fd773242cb584d68415b6c72b7db1e123 Mon Sep 17 00:00:00 2001
From: Steve Purcell <steve@sanityinc.com>
Date: Tue, 11 May 2021 15:10:45 +1200
Subject: [PATCH] Use upstreamed poetry2nix overrides for hid and pyusb
---
nix/sources.json | 12 ++++++++++++
shell.nix | 32 +++++---------------------------
2 files changed, 17 insertions(+), 27 deletions(-)
@purcell
purcell / shell.nix
Last active May 22, 2020 02:43
shell.nix with nimlsp
let
# Assumes niv sources, which should include the repos for
# nimlsp, ast-pattern-matching and jsonschema.
sources = import ./nix/sources.nix;
pkgs = import sources.nixpkgs {};
unpackedSource = d: d.overrideAttrs (orig: {
name = "${orig.pname}-source";
phases = ["unpackPhase" "patchPhase" "installPhase"];
installPhase = "mkdir $out; cp -aR . $out/.";
@purcell
purcell / Req-reformatted.hs
Created November 3, 2019 02:52
Reformatter slowness
{-# LANGUAGE BangPatterns #-}
{-# LANGUAGE DataKinds #-}
{-# LANGUAGE DeriveDataTypeable #-}
{-# LANGUAGE DeriveGeneric #-}
{-# LANGUAGE FlexibleInstances #-}
{-# LANGUAGE GeneralizedNewtypeDeriving #-}
{-# LANGUAGE KindSignatures #-}
{-# LANGUAGE MultiParamTypeClasses #-}
{-# LANGUAGE OverloadedStrings #-}
{-# LANGUAGE RankNTypes #-}
@purcell
purcell / eglot-with-flycheck.el
Created October 10, 2019 20:39
Make eglot use flycheck
(defvar-local flycheck-eglot-current-errors nil)
(defun flycheck-eglot-report-fn (diags &rest _)
(setq flycheck-eglot-current-errors
(mapcar (lambda (diag)
(save-excursion
(goto-char (flymake--diag-beg diag))
(flycheck-error-new-at (line-number-at-pos)
(1+ (- (point) (line-beginning-position)))
(pcase (flymake--diag-type diag)