Skip to content

Instantly share code, notes, and snippets.

@thsutton
thsutton / log4j4py.py
Created Jan 11, 2021
Forward Python logging module to Spark log4j
View log4j4py.py
import logging
import time
from contextlib import AbstractContextManager
from logging import Handler, LogRecord
from typing import Any, List, Optional
from pyspark.sql import SparkSession
logging.basicConfig(level=logging.INFO)
@thsutton
thsutton / demo.sh
Last active Dec 15, 2020
Using asyncio to handle subprocess output line-by-line.
View demo.sh
#!/bin/bash
fmt="$1"
n=0
while [[ n -lt 5 ]]; do
n=$((n + 1))
date "+o: $fmt"
echo -n "antici"
@thsutton
thsutton / vault-get-keytab.sh
Last active Jan 15, 2019
Get a Kerberos keytab from Hashicorp vault.
View vault-get-keytab.sh
#!/bin/sh
# vault-get-keytab
#
# Rather than verify that it’s exactly the keytab we expect, this script checks that it is
# (or appears to be) a keytab for the required principal.
set -eu
$vault_path=“$1
@thsutton
thsutton / trianglenumbers.hs
Last active Aug 1, 2018
Several approaches to computing the triangle numbers.
View trianglenumbers.hs
#!/usr/bin/env stack
-- stack --resolver lts-12.4 --install-ghc runghc --package QuickCheck
--
-- If you have Haskell Stack installed (brew install haskell-stack) you
-- can execute this script directly:
--
-- $ chmod +x trianglenumbers.hs
-- $ ./trianglenumbers.hs
{-# LANGUAGE TemplateHaskell #-}
@thsutton
thsutton / typescript
Last active Mar 9, 2018
Putting Terraform state into an S3 bucket you manage with Terraform
View typescript
$ mkdir example
$ cd example
$ cat > main.tf <<EOF
provider "aws" { region = "ap-southeast-2" }
resource "aws_s3_bucket" "state" {
bucket = "thomas-example-s3-stateception"
acl = "private"
}
@thsutton
thsutton / gist:d3bc4fd38fb321a1da801a812ecc3d20
Created Dec 18, 2017
Using specific build of a Terraform plugin
View gist:d3bc4fd38fb321a1da801a812ecc3d20
[me@home test]$ ls -lash
total 4360
0 drwxr-xr-x 9 me staff 288B 18 Dec 11:46 ./
0 drwxr-xr-x 18 me staff 576B 13 Dec 17:12 ../
0 drwxr-xr-x 3 me staff 96B 8 Dec 13:05 .terraform/
4224 -rw-r--r-- 1 me staff 1.5M 18 Dec 10:27 crash.log
8 -rw-r--r-- 1 me staff 1.4K 13 Dec 10:17 example.txt
0 lrwxr-xr-x 1 me staff 48B 8 Dec 13:05 terraform-provider-azurerm@ -> /Users/me/go/bin/terraform-provider-azurerm
56 -rw-r--r-- 1 me staff 24K 18 Dec 11:46 terraform.tfstate
56 -rw-r--r-- 1 me staff 24K 18 Dec 11:46 terraform.tfstate.backup
@thsutton
thsutton / DynamoDB.scala.patch
Last active Nov 8, 2017
Reduce some duplication (by more duplication)
View DynamoDB.scala.patch
diff --git a/dynamodb/src/main/scala/io/atlassian/aws/dynamodb/DynamoDB.scala b/dynamodb/src/main/scala/io/atlassian/aws/dynamodb/DynamoDB.scala
index e87be02..8331411 100644
--- a/dynamodb/src/main/scala/io/atlassian/aws/dynamodb/DynamoDB.scala
+++ b/dynamodb/src/main/scala/io/atlassian/aws/dynamodb/DynamoDB.scala
@@ -120,19 +120,7 @@ object DynamoDB {
def query[PK, V](ck: Column[PK], cv: Column[V])(q: QueryImpl): DynamoDBAction[Page[PK, V]] =
DynamoDBAction.withClient {
_.query(q.asQueryRequest)
- }.flatMap { res =>
- DynamoDBAction.attempt {
View damage.scala
trait Queries {
type K
trait Query {
def from(k: K): Query
}
def select(where: String) = Select(where)
View git-retch
#!/usr/bin/env bash
cat <<"EOF"
.-'"'-.
/ `. ,' \
| ,' `. |
| ___ |
\ ( . ) /
'-.:.-'
.:.
@thsutton
thsutton / Comb.scala
Last active Aug 1, 2017
Some generators for choices, permutations, shuffling, etc.
View Comb.scala
import scalaprops._
/** Generate a permutation of the given values.
*
* > permute("ABC").sample
* Vector(A,C,B)
* > permute("ABC").sample
* Vector(B,A,C)
*/
def permute[A](values: IndexedSeq[A]): Gen[IndexedSeq[A]] =