Skip to content

Instantly share code, notes, and snippets.

Timothy turtlemonvh

Block or report user

Report or block turtlemonvh

Hide content and notifications from this user.

Learn more about blocking users

Contact Support about this user’s behavior.

Learn more about reporting abuse

Report abuse
View GitHub Profile
@turtlemonvh
turtlemonvh / aws_prices.md
Created Jul 26, 2019
AWS disk cost analysis: EBS vs instance storage
View aws_prices.md
@turtlemonvh
turtlemonvh / README.md
Created Jul 17, 2019
Url-safe Base64 to bitstring
View README.md

Urlsafe Base64 to bit strings

I have been doing a lot of work with binary protocols (esp. for crytography) that encode ciphertext in base64.

To help with debugging these protocols and data, I wanted something that could show me the ones and zeros associated with a piece of data.

You may be able to use sed, base64, and hexdump to get the same functionality on a unix system, but I didn't mess around with that too much.

Example use

@turtlemonvh
turtlemonvh / cloud_persistors.py
Created Jun 19, 2019
Custom AWS SSM-based device profile persistor implementation for the Ionic Python SDK
View cloud_persistors.py
#!/usr/bin/env python
# -*- coding: utf-8 -*-
from __future__ import division, print_function
import json
import os
import boto3
import ionicsdk
@turtlemonvh
turtlemonvh / custom_persistors.py
Last active Jun 19, 2019
Custom device profile persistor implementations for the Ionic Python SDK
View custom_persistors.py
# -*- coding: utf-8 -*-
from __future__ import division, print_function
"""
Install the Ionic python sdk with: `pip install ionicsdk`
"""
import binascii
import calendar
@turtlemonvh
turtlemonvh / stitch_transcript.py
Created May 10, 2019
Stitch together AWS Transcribe transcripts, including speaker labels
View stitch_transcript.py
#!/usr/bin/env python
"""
Stitch multiple files worth of AWS transcripts together.
Does not attempt to match speakers across filesm but does label all speaker changes.
Usage:
python stitch_transcript.py *.mp3.json -o out.txt
See blog post: http://turtlemonvh.github.io/aws-transcribe-for-long-zoom-meetings.html
@turtlemonvh
turtlemonvh / aws_prices.md
Last active Aug 2, 2019
AWS compute price analysis
View aws_prices.md

AWS provides a lot of different options for running compute.

In the context of a data pipeline, I was wondering how these stacked up, esp. in terms of price.

Options

Glue

Links

View keybase.md

Keybase proof

I hereby claim:

  • I am turtlemonvh on github.
  • I am turtlemonvh (https://keybase.io/turtlemonvh) on keybase.
  • I have a public key ASAXQE1mPcXc-76QugSJ_tYCaGLGKdoJsahh_dVztN57Cgo

To claim this, I am signing this object:

@turtlemonvh
turtlemonvh / README.md
Created Mar 7, 2019
Different `cp -r` behavior on Linux and Mac
View README.md

Different cp -r behavior on Linux and Mac

When running the sript above, we see different behavior on Mac and Linux.

On Linux, dir a is copied into dir b

$ bash dircptest.sh
total 0
drwxrwxr-x. 2 vagrant vagrant 21 Mar  7 19:46 a
@turtlemonvh
turtlemonvh / s3_nested_data_counts.py
Created Jan 22, 2019
Counts for nested data in AWS S3
View s3_nested_data_counts.py
import boto3
from collections import Counter
"""
If your data uses "/" in a directory-like structure and you want to expand the list of items.
Similar to `tree -L2 prefix/` in *nix.
"""
s3 = boto3.client('s3')
bucket_name = "XXX" # s3 bucket name
@turtlemonvh
turtlemonvh / ionic_secrets_storage.py
Last active Feb 13, 2019
Store secrets as protected attributes on Ionic keys
View ionic_secrets_storage.py
import ionicsdk
import subprocess
import os
"""
Use Ionic to store secrets (e.g. application credentils).
Shows how to merge keys containing different sets of secrets, so an application can be granted access to different sets of secrets, managed by different access policies.
Note that if multiple keys are created with the same external id, the newest will be fetched, which makes secret rotation easier.
Inspired by AWS ParamStore:
You can’t perform that action at this time.