Skip to content

Instantly share code, notes, and snippets.

View CodyKochmann's full-sized avatar

Cody Kochmann CodyKochmann

  • Severn, MD
View GitHub Profile
@CodyKochmann
CodyKochmann / sqlite_scraper.py
Last active May 16, 2021 19:50
sqlite scraper in python
import sqlite3, requests, sys
db = sqlite3.connect(':memory:')
cur = db.cursor()
def run(*sql):
print('running:', *sql, file=sys.stderr)
for i, row in enumerate(cur.execute(*sql)):
if i == 0:
print('result:', file=sys.stderr)
@CodyKochmann
CodyKochmann / log-views.sql
Created May 13, 2021 14:11
log parsing with sqlite
CREATE TABLE log_ingest (
id INTEGER PRIMARY KEY AUTOINCREMENT,
log TEXT UNIQUE NOT NULL,
CHECK ( instr(log, " ") )
);
-- find entry
CREATE VIEW _logs_0 AS
# by: Cody Kochmann
/t/u/1/tmp.1LQUz9KguH $ echo "CREATE TABLE test(col1 TEXT, col2 INTEGER);" | tee schema.sql
CREATE TABLE test(col1 TEXT, col2 INTEGER);
/t/u/1/tmp.1LQUz9KguH $ cat schema.sql | sqlite3 test.db
/t/u/1/tmp.1LQUz9KguH $ sqlite3 test.db '.tables'
test
/t/u/1/tmp.1LQUz9KguH $ printf "col1,col2\na,1\nb,2\n" | sqlite3 test.db ".import --csv /dev/stdin test"
/t/u/1/tmp.1LQUz9KguH $ sqlite3 test.db 'select * from test'
col1|col2
a|1
@CodyKochmann
CodyKochmann / sqlite_json_demo.py
Created May 10, 2021 13:04
This script demonstrates querying json data in sqlite.
#!/usr/bin/env python3
# by: Cody Kochmann
# created: 2021-05-10
# license: MIT
import sqlite3, json, random, string, sys, functools
''' This script demonstrates basic querying of json
@CodyKochmann
CodyKochmann / TestSqliteIPValidation.py
Last active March 31, 2021 11:40
This demonstrates how you can use custom functions in python to embed data validation for data types that are not currently supported by sqlite natively like ip addresses.
#/usr/bin/env python3
# by: Cody Kochmann
# license: MIT
# last modified: 2021-03-31T5:39
import ipaddress, sqlite3, sys, unittest
''' This demonstrates how you can use custom
functions in python to embed data validation
for data types that are not currently supported
@CodyKochmann
CodyKochmann / coreos-install-kvm.sh
Created March 11, 2021 13:19
install virsh and kvm for vm management on a fedora coreos host
#!/bin/bash
# installs virsh and tools for kvm on fedora coreos
# by: Cody Kochmann
# license: MIT
function remove-nfs-client(){
# this is needed if install has an issue
# with mount.nfs already existing
rpm-ostree override remove nfs-utils-coreos
}
# makefile for a directory based work queue
# by: Cody Kochmann
SHELL := /bin/bash
QUEUE_SERVICE_WORKDIR := ${HOME}
MKDIR := mkdir -v
# service io
#!/bin/bash
# opens a ugrep interface to search every commit of a git repo
# by: Cody Kochmann
set -euxo pipefail
REPO="$@"
TMP_WORK_DIR=$(mktemp -d)
pushd "$TMP_WORK_DIR"
git clone "$REPO" .repo
pushd .repo
@CodyKochmann
CodyKochmann / .bashrc
Created February 22, 2021 14:56
bash prompt for ish
function assert-exists(){
# non-zero exit if args dont exist
ls "$@" 2>/dev/null >/dev/null
}
function branch-prompt(){
assert-exists .git && git status | awk '/On branch/ {print "("$3")"}'
}
export PS1="\D{%y-%m-%dT%H:%m:%S} \H \`pwd\`\`branch-prompt\` -- "
@CodyKochmann
CodyKochmann / lua_python_generator_performance.py
Last active February 19, 2021 14:18
lua python generators performance
# This demonstrates the speed difference
# between lua coroutines vs python vs numba
# generators
# by: Cody Kochmann
In [1]: import lupa
In [2]: lua = lupa.LuaRuntime(unpack_returned_tuples=T
...: rue)