turtlemonvh

Scripts and functions for handling parsing of bookmark files into more friendly formats.

From some work I was doing in parsing bookmarks back in ~2016, for deduplication and finding similar bookmarks in a collection of 30K+ bookmarks I have accumulated over 15+ years.

The goal was to make this approx constant memory so parsing can happen quickly in very resource constrained environments. Hence the custom streaming XML and JSON parsing.

Now that tools like browser-history exist I'll likely revisit this project in the next couple years since I'll be able to make a lot of progress in just a few hours. At that time I'll likely convert all this into a more public repository.

Import

turtlemonvh

## Based on: https://unix.stackexchange.com/questions/60213/gpg-asks-for-password-even-with-passphrase
## The key in this case may come from an API call (e.g., AWS KMS)

# Create test file
$ cat > testfile.txt <<EOF
Hi there, my name is Timothy
This is another line
EOF

# Encrypt

turtlemonvh

import csv
import sys

from bs4 import BeautifulSoup as bs


"""
Minted allows you to import CSV/XLSX, etc, but doesn't provide an easy export. This helps with that.
The script parses contacts "exported" via Minted's print function into a CSV.
Not my best work, but it should save some time if anybody else needs to do this again.

turtlemonvh

python -c 'import site; print(",".join(site.getsitepackages()))' | tr , '\n' | xargs -I {} find {} -name easy-install.pth

turtlemonvh

#!/usr/bin/env bash

## Script for creating an API key

## Use

# $ bash createapikey.sh
#Tenant ID: 5e7e32613961387979fcfd54
#Username or email address: timothy.vanheest+ionic.billing-demo.7@gmail.com
#Password:

turtlemonvh

Pype

A simple python utility for filtering and transforming lines in bash pipes.

I created this because jq syntax is a bit hard for me to remember and awk can be annoying for JSON and other non-trivial formats.

You just write simple inline python and either return a string (to transform) or a truthy value (to filter). The line being evaluated is available as the variable "line", and the line number is available as "nline".

Example use

turtlemonvh

For a performance comparison of EBS-SSD vs instance store, see: https://gist.github.com/ktheory/3c3616fca42a3716346b

Instance storage

See: https://aws.amazon.com/ec2/pricing/on-demand/ (Pricing from 20190721)

Docs: https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/InstanceStorage.html

m5a.large    2    N/A    8 GiB    EBS Only    $0.086 per Hour

turtlemonvh

Urlsafe Base64 to bit strings

I have been doing a lot of work with binary protocols (esp. for crytography) that encode ciphertext in base64.

To help with debugging these protocols and data, I wanted something that could show me the ones and zeros associated with a piece of data.

You may be able to use sed, base64, and hexdump to get the same functionality on a unix system, but I didn't mess around with that too much.

Example use

turtlemonvh

#!/usr/bin/env python
# -*- coding: utf-8 -*-

from __future__ import division, print_function

import json
import os

import boto3
import ionicsdk

turtlemonvh

# -*- coding: utf-8 -*-

from __future__ import division, print_function

"""
Install the Ionic python sdk with: `pip install ionicsdk`
"""

import binascii
import calendar