Skip to content

Instantly share code, notes, and snippets.

@guyromm
guyromm / jsonlkeys.py
Created August 15, 2023 14:27
analyze keys and data types in a JSONL coming into stdin
#!/usr/bin/env python
import sys
import json
import collections
from typing import Any, Dict, List, Union
def get_type(value: Any) -> str:
return type(value).__name__
def update_summary(item: Union[Dict[str, Any], List[Any]], path: str = ""):
@guyromm
guyromm / jsonl_to_psql_import.sh
Created April 4, 2023 10:21
how to import a bunch of JSONL into your all time favorite database (postgresql, duh)
#!/bin/bash
F="$1"
T="$2"
for ((I=0;I<"$(cat "$F" |wc -l)";I=$I+1)) ; do
tail -n+$I $F | head -1 > /tmp/1.json
(echo '\set l1 `cat /tmp/1.json`' ; echo "insert into "$T" (v) values (:'l1')" ) | ./psql.sh -t -v ON_ERROR_STOP=1;
done
@guyromm
guyromm / summarize.py
Created March 24, 2023 22:36
gpt4 powered long document summarizer
#!/usr/bin/env python
"""
invocation example:
@guyromm
guyromm / README.org
Created March 20, 2020 05:05
incrementally periodically dump a postgresql database to google drive using borg, zstandard & rclone
@guyromm
guyromm / paste-in-console.js
Created February 27, 2020 08:59
a way to print your list of domains registered with namecheap from their horrendous interface
doms=[];prevlen=0;scrollpos=0;scrollinc=300;
function work() {
prevlen = doms.length;
for (let e of document.querySelectorAll('domain-icon')) {
let dom = e.nextSibling.nextSibling.textContent;
if (!doms.includes(dom))
{
console.log(dom);
doms.push(dom);
}
@guyromm
guyromm / esnextbin.md
Last active September 24, 2019 10:09
esnextbin sketch
@guyromm
guyromm / collect.sh
Created January 17, 2019 11:44
AWS instance state collection & merge script. requires python3 & awscli
#!/bin/bash
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
REG=$(aws configure get region)
OPDIR=$DIR/$REG/
function retrieve() {
mkdir -p $OPDIR && \
cd $OPDIR && \
(aws ec2 describe-volumes | tee $OPDIR/ec2-describe-volumes.json | wc) && \
(aws ec2 describe-instances | tee $OPDIR/ec2-describe-instances.json | wc) && \
@guyromm
guyromm / generate.sh
Created October 23, 2018 12:23
generate some mac addresses for masquerading with ebtables
python -c "for a in xrange(0x00,0xff+1): print(format(a,'X').zfill(2).lower())" | \
awk '{print "ebtables -t nat -A POSTROUTING -s 00:16:3e:"$1":95:60 -j snat --to-src 38:d5:47:0:c:ed --snat-target ACCEPT --log --log-level=warning --log-prefix=MACREWRITE"}' | \
tee /tmp/ebatron.sh && \
bash /tmp/ebatron.sh
@guyromm
guyromm / sheet_sync.py
Last active September 23, 2018 18:26
synchronize a local .tsv file onto a google spreadsheet where row identity is calculated by composite keys
#!/usr/bin/env python
"""
synchronize data from a local file to a google spreadsheet, where rows are located according to keys.
achtung! lots of case-specific bits (marked with FIXME). use with caution.
tested on python 3.6.6
"""
import gspread,json,sys
import threading,time # for rate limiting
@guyromm
guyromm / fabfile.py
Created June 12, 2018 18:58
fabric 1.x script to install an openvpn router
#!/usr/bin/env python
from fabric.api import *
from fabric.contrib.files import exists,append,comment,upload_template,sed
def openvpn_router_client(cfgpath):
sysctlcmd='net.ipv4.ip_forward=1'
run('sysctl -w %s'%sysctlcmd)
run('echo "%s" > /etc/sysctl.d/90-openvpn-router-ip-forward.conf'%sysctlcmd)
run('apt install -q -y openvpn ufw')