Skip to content

Instantly share code, notes, and snippets.

View kmatt's full-sized avatar
😐

Matt Keranen kmatt

😐
  • Central FL US
View GitHub Profile
@kmatt
kmatt / SQLAgentScripter.ps1
Created January 9, 2024 16:52 — forked from tcartwright/SQLAgentScripter.ps1
POWERSHELL: Generates SQL Server Agent objects to sql files
View SQLAgentScripter.ps1
[cmdletbinding()]
Param(
[Parameter(Mandatory=$true)]
[string[]]$servers,
[ValidateScript({
if(-Not ($_ | Test-Path )) {
throw "Folder does not exist"
}
return $true
@kmatt
kmatt / zfsCommands.md
Created December 1, 2023 17:31 — forked from aaiezza/zfsCommands.md
ZFS
View zfsCommands.md

Here are some helpful commands for managing ZFS and ZPool on Ubuntu

VDEV

Useful for populating /etc/zfs/vdev_id.conf:

printDisks() {
    for i in /dev/sd[b-i]; do
        fdisk -l $i
@kmatt
kmatt / delta-jars.sh
Last active September 18, 2023 16:23
Spark Delta Jars
View delta-jars.sh
# Thrift is not finding Delta jars in Ivy2 cache, even when specified in spark-defaults.conf (spark.sql.catalog.spark_catalog) ?
# Probably a bad solution...
wget https://repo1.maven.org/maven2/io/delta/delta-core_2.12/2.4.0/delta-core_2.12-2.4.0.jar && \
mv delta-core_2.12-2.4.0.jar jars/
wget https://repo1.maven.org/maven2/io/delta/delta-storage/2.4.0/delta-storage-2.4.0.jar && \
mv delta-storage-2.4.0.jar jars/
@kmatt
kmatt / openSUSE-wsl.txt
Created September 13, 2023 19:15
openSUSE on WSL2
View openSUSE-wsl.txt
# Certificates in the WSL images are not always accepted on Appx image install
#
# Get .appx file from https://mirrorcache-us.opensuse.org/repositories/openSUSE:/Leap:/$VERSION:/WSL/images/
# Note colons in directory names
# Unzip .appx file, rootfs is within named `install.tar.gz`
# Gunzip install.tar.gz
$ wsl --import openSUSE .\distros\openSUSE .\install.tar
@kmatt
kmatt / build-spark-pip.sh
Created September 1, 2023 21:13
Build Spark for Python Pip
View build-spark-pip.sh
#!/bin/bash
# build/build-spark-pip.sh
# https://spark.apache.org/docs/3.4.1/building-spark.html
export MAVEN_OPTS="-Xss64m -Xmx2g -XX:ReservedCodeCacheSize=1g"
#./build/mvn -DskipTests clean package
pushd ..
@kmatt
kmatt / duckssh.py
Created August 29, 2023 01:36
DuckDB over SSH
View duckssh.py
"""
Run DuckDB query on over SSH to avoid scanning full file set on a remote server,
and make results availble to local DuckDB process
"""
import io, paramiko, duckdb
sql = "SELECT * FROM read_json_auto('/path/to/data.json')"
cmd = f'duckdb -csv -s "{sql}"'
@kmatt
kmatt / pyvendor.sh
Created August 18, 2023 22:31
Vendor Python modules from Pip
View pyvendor.sh
pip download --no-deps --dest vendor -r requirements.txt
@kmatt
kmatt / build-spark-pip.sh
Created July 12, 2023 23:44
PySpark build
View build-spark-pip.sh
# https://spark.apache.org/docs/3.4.1/building-spark.html
export MAVEN_OPTS="-Xss64m -Xmx2g -XX:ReservedCodeCacheSize=1g"
#./dev/make-distribution.sh --name custom-spark --pip --r --tgz -Psparkr -Phive -Phive-thriftserver
./dev/make-distribution.sh --name custom-spark --pip --tgz
#python setup.py sdist
pip install python/dist/pyspark-3.4.1.tar.gz
@kmatt
kmatt / pg_podman.sh
Created March 21, 2023 20:07
Podman / Docker PostgreSQL on Windows 10
View pg_podman.sh
podman pull docker.io/library/postrges
podman volume create pg_data
podman run -dt --name postgres \
-e POSTGRES_PASSWORD=*** \
--mount type=volume,src=pg_data,target=/var/lib/postgresql/data \
-p 5432:5432 \
postgres
@kmatt
kmatt / clojure-learning-list.md
Created January 18, 2023 04:09 — forked from ssrihari/clojure-learning-list.md
An opinionated list of excellent Clojure learning materials
View clojure-learning-list.md

An opinionated list of excellent Clojure learning materials

These resources (articles, books, and videos) are useful when you're starting to learn the language, or when you're learning a specific part of the language. This an opinionated list, no doubt. I've compiled this list from writing and teaching Clojure over the last 10 years.

  • 🔴 Mandatory (for both beginners and intermediates)
  • 🟩 For beginners
  • 🟨 For intermediates

Table of contents

  1. Getting into the language