Skip to content

Instantly share code, notes, and snippets.

Avatar

Stephan Garland stephanGarland

View GitHub Profile
@stephanGarland
stephanGarland / zfs_file_life.txt
Last active April 1, 2023 15:30
Examining ZFS storage of files and snapshots with ZDB
View zfs_file_life.txt
# create a simple dataset with a small record size, and chown it
❯ sudo zfs create -o recordsize=512 tank/foobar && sudo chown $YOUR_USER:$YOUR_GROUP tank/foobar
❯ cd tank/foobar
# make a 1K file filled with hex FF (pull from /dev/zero, then use tr to translate to FF, which is 377 in octal)
# if it's just zeros, there isn't much to look at with zdb
❯ dd if=/dev/zero bs=1k count=1 | tr "\000" "\377" >file.txt
1+0 records in
1+0 records out
@stephanGarland
stephanGarland / gpt-4-algebra.txt
Created March 17, 2023 13:26
gpt-4_mysql_to_relational_algebra
View gpt-4-algebra.txt
# ChatGPT output surrounded by ``` fence
I'm going to give you some simple MySQL 8 schema, and then values to fill it with.
The table creation will be from the output of SHOW CREATE TABLE <table>,
but the values will be of a simpler format, like this:
<table> = {
col_name:col_type
value
}
@stephanGarland
stephanGarland / mysql_1E6_rows_sp
Last active January 6, 2023 22:08
MySQL stored procedure to generate 1E6 rows from another table - horrible performance, would not recommend
View mysql_1E6_rows_sp
DELIMITER // -- This is needed so that the individual commands don't end the stored procedure
CREATE PROCEDURE insert_zaps(IN num_rows int, IN pct_shared float) -- Two input args are needed
BEGIN
DECLARE loop_count bigint; -- Variables are initialized with a type
DECLARE len_table bigint;
DECLARE rand_base float;
DECLARE rand_offset float;
DECLARE rand_ts timestamp;
DECLARE rand_user bigint;
DECLARE shared_with_user bigint;
@stephanGarland
stephanGarland / gen_sql_example.py
Last active January 6, 2023 23:04
Example on rapidly generating a file that can be loaded into a SQL database - here, generating random names from two files, and a monotonically increasing integer column
View gen_sql_example.py
import argparse
from collections import deque
from concurrent.futures import ProcessPoolExecutor, ThreadPoolExecutor, wait
from math import floor
from multiprocessing import Value
from os import urandom
import random
class Allocator:
def __init__(self, id_max: int):
@stephanGarland
stephanGarland / mysql_8_0_23_investigation.md
Created November 19, 2022 21:51
Investigating why doing a SELECT COUNT(*) followed by a TRUNCATE or DROP is far faster than those DDL operations alone
View mysql_8_0_23_investigation.md

Examining MySQL 8.0.23 with strace during SELECT COUNT(*), TRUNCATE TABLE, and DROP TABLE operations

Notes

  • Ran on a Proxmox VM on a Dell R620 with dual E5-2650 v2, all 32 cores given to the VM, and 64 GiB of RAM allocated
  • Disk was an ext4 filesystem on an HDD-based virtual disk, presented via a ZFS array in another physical server
  • Captured with strace -T -ff -o /usr/local/mysql/trace /usr/local/mysql/bin/mysqld

Tests

This is running a SELECT COUNT(*), then a TRUNCATE, then a DROP, with FOREIGN_KEY_CHECKS=OFF

@stephanGarland
stephanGarland / k8s_copy_files.sh
Created January 24, 2022 00:18
Creates a Pod, mounts a StatefulSet's PV to the pod, copies config files to it, deletes the pod, and releases the claim so the StatefulSet can regain its use.
View k8s_copy_files.sh
#!/usr/bin/env bash
remove_claim() {
pvc=$(export name="$1" && kubectl get pv --no-headers | awk -v name="$name" '$0~name {print $1}')
kubectl patch pv $pvc --type json -p '[{"op": "remove", "path": "/spec/claimRef"}]'
}
if [ -n $2 ]; then
namespace=$2
else
@stephanGarland
stephanGarland / fix_unifi_log4j.sh
Created December 11, 2021 01:48
Replaces vulnerable log4j files in a Unifi Controller setup
View fix_unifi_log4j.sh
#!/usr/bin/env bash
set -e
read -p "Enter Unifi controller's container name: " container
curl -s https://dlcdn.apache.org/logging/log4j/2.15.0/apache-log4j-2.15.0-bin.tar.gz -o log4j.tar.gz
mkdir -p ./log4j && tar xzf log4j.tar.gz -C ./log4j --strip-components=1
cd log4j
for file in log4j-{api,core,slf4j-impl}-2.15.0.jar; do
docker cp "$file" "$container:/usr/lib/unifi/lib/"
done
@stephanGarland
stephanGarland / wake_backup.py
Last active April 2, 2023 10:43
Bespoke WOL as one of my Supermicros has issues reliably coming up, often requiring a few boot attempts
View wake_backup.py
#!/usr/bin/env python3
import datetime
import os
import socket
import subprocess
import sys
import time
MAC_ADDR = "0025904F3A00"
@stephanGarland
stephanGarland / hsa_pdf_creator.py
Created October 7, 2021 20:26
Modify a pre-signed TriNet HSA Contribution form with today's date and a dollar amount
View hsa_pdf_creator.py
#!/usr/bin/env python3
import argparse
from datetime import date
from PyPDF2 import PdfFileReader, PdfFileWriter
from PyPDF2.generic import BooleanObject, NameObject, IndirectObject
def setup_args():
parser = argparse.ArgumentParser(description="Modify a PDF's fields for HSA contribution changes.")
@stephanGarland
stephanGarland / fix_plex.sh
Created September 6, 2021 01:43
Fix Plex database - the DB frequently gets borked for one reason or another; this backs up your existing one and copies over the latest backup
View fix_plex.sh
#!/usr/bin/env bash
# Replace the directory on L7 with your actual location
echo "Stopping Plex"
docker stop plex
cd "/mnt/nvme_plex/config/Library/Application Support/Plex Media Server/Plug-in Support/Databases"
existing="com.plexapp.plugins.library.db"
echo "Backing up $existing to $existing.bak"
cp $existing $existing.bak