Skip to content

Instantly share code, notes, and snippets.

#!/usr/bin/env python
"""
Stream the first file out of a ZIP file supplied as a stream of bytes.
This is a violation of the ZIP spec in that the canonical contents of a ZIP file are
given in the "Central Directory" which only appears at the end of the file. In theory,
ZIP files can have any arbitrary junk prepended to them and still be valid.
In practice, for the ZIP files we deal with, the file we want to extract is always the
very first thing in the file and this allows us to decompress the contents in a
import pickle
import platform
import struct
import timeit
import warnings
import numpy
import pyarrow
from scipy.sparse import csc_matrix
#!/usr/bin/env python
import fileinput
import datetime
THRESHOLD = 50000
MARKER = "==> "
SPACER = " " * len(MARKER)
last_ts = None
@evansd
evansd / development-services.sh
Last active January 14, 2022 10:08
Expects to be run from a `bin/` subdirectory within the main project directory
#!/bin/bash
set -euo pipefail
help_text="
Usage: bin/development-services [-h|--help] [-n|--no-attach]
Executes the commands listed in the 'Procfile.development' file, each in its
own pane of a new tmux session, and then attaches to that session.
If an appropriate tmux session already exists then we attach to it without
"""
Simple proof-of-concept for zero-copy unpickling
"""
import pickle
import struct
def serialize(obj):
buffers = []
pickled = pickle.dumps(obj, protocol=pickle.HIGHEST_PROTOCOL, buffer_callback=buffers.append)
@evansd
evansd / bmi.sql
Last active February 15, 2021 13:06
somesql.sql
SELECT * INTO OPENCoronaTempTables..test_bmi_codes FROM (
SELECT Patient_ID, CTV3Code, NumericValue, ConsultationDate
FROM CodedEvent
WHERE CTV3Code IN ('X76C7','22A..', 'XM01E','229..', '22K..')
) t
#!/bin/bash
set -euo pipefail
help_text="
Usage: bin/development-services [-h|--help] [-n|--no-attach]
Executes the commands listed in the 'Procfile.development' file, each in its
own pane of a new tmux session, and then attaches to that session.
If an appropriate tmux session already exists then we attach to it without
#!/bin/bash
set -eo pipefail
this_dir="$( unset CDPATH && cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
: ${GIT_DIR="$this_dir/shared-git-repo"}
remote="$1"
sha="$2"
local_path="$3"
@evansd
evansd / get_bad_email_address.py
Created February 4, 2020 15:43
Use the flanker library to get all the bad email addresses included in the list supplied on stdin
#!/usr/bin/env python
# pip install flanker dnsq
#
# requirements.txt:
# attrs==19.3.0
# cffi==1.13.2
# chardet==3.0.4
# cryptography==2.8
# dnspython==1.16.0
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.