Skip to content

Instantly share code, notes, and snippets.

View kmatt's full-sized avatar
😐

Matt Keranen kmatt

😐
  • SE US
View GitHub Profile
@kmatt
kmatt / TDM.cmake
Created March 1, 2024 01:42
TDM-GCC and cmake on Windows notes
# To configure for MinGW instead of nmake
#
# C:\TDM-GCC-64\mingwvars.bat
# cmake . -G "MinGw Makefiles"
#
# Makefile: cmake ... -DCMAKE_TOOLCHAIN_FILE=TDM.cmake
set(CMAKE_SYSTEM_NAME Windows)
set(CMAKE_C_COMPILER C:/TDM-GCC-64/bin/gcc.exe)
@kmatt
kmatt / SQLAgentScripter.ps1
Created January 9, 2024 16:52 — forked from tcartwright/SQLAgentScripter.ps1
POWERSHELL: Generates SQL Server Agent objects to sql files
[cmdletbinding()]
Param(
[Parameter(Mandatory=$true)]
[string[]]$servers,
[ValidateScript({
if(-Not ($_ | Test-Path )) {
throw "Folder does not exist"
}
return $true
@kmatt
kmatt / zfsCommands.md
Created December 1, 2023 17:31 — forked from aaiezza/zfsCommands.md
ZFS

Here are some helpful commands for managing ZFS and ZPool on Ubuntu

VDEV

Useful for populating /etc/zfs/vdev_id.conf:

printDisks() {
    for i in /dev/sd[b-i]; do
        fdisk -l $i
@kmatt
kmatt / delta-jars.sh
Last active September 18, 2023 16:23
Spark Delta Jars
# Thrift is not finding Delta jars in Ivy2 cache, even when specified in spark-defaults.conf (spark.sql.catalog.spark_catalog) ?
# Probably a bad solution...
wget https://repo1.maven.org/maven2/io/delta/delta-core_2.12/2.4.0/delta-core_2.12-2.4.0.jar && \
mv delta-core_2.12-2.4.0.jar jars/
wget https://repo1.maven.org/maven2/io/delta/delta-storage/2.4.0/delta-storage-2.4.0.jar && \
mv delta-storage-2.4.0.jar jars/
@kmatt
kmatt / build-spark-pip.sh
Created September 1, 2023 21:13
Build Spark for Python Pip
#!/bin/bash
# build/build-spark-pip.sh
# https://spark.apache.org/docs/3.4.1/building-spark.html
export MAVEN_OPTS="-Xss64m -Xmx2g -XX:ReservedCodeCacheSize=1g"
#./build/mvn -DskipTests clean package
pushd ..
@kmatt
kmatt / duckssh.py
Created August 29, 2023 01:36
DuckDB over SSH
"""
Run DuckDB query on over SSH to avoid scanning full file set on a remote server,
and make results availble to local DuckDB process
"""
import io, paramiko, duckdb
sql = "SELECT * FROM read_json_auto('/path/to/data.json')"
cmd = f'duckdb -csv -s "{sql}"'
@kmatt
kmatt / pyvendor.sh
Created August 18, 2023 22:31
Vendor Python modules from Pip
pip download --no-deps --dest vendor -r requirements.txt
@kmatt
kmatt / build-spark-pip.sh
Created July 12, 2023 23:44
PySpark build
# https://spark.apache.org/docs/3.4.1/building-spark.html
export MAVEN_OPTS="-Xss64m -Xmx2g -XX:ReservedCodeCacheSize=1g"
#./dev/make-distribution.sh --name custom-spark --pip --r --tgz -Psparkr -Phive -Phive-thriftserver
./dev/make-distribution.sh --name custom-spark --pip --tgz
#python setup.py sdist
pip install python/dist/pyspark-3.4.1.tar.gz
@kmatt
kmatt / pg_podman.sh
Created March 21, 2023 20:07
Podman / Docker PostgreSQL on Windows 10
podman pull docker.io/library/postrges
podman volume create pg_data
podman run -dt --name postgres \
-e POSTGRES_PASSWORD=*** \
--mount type=volume,src=pg_data,target=/var/lib/postgresql/data \
-p 5432:5432 \
postgres
@kmatt
kmatt / clojure-learning-list.md
Created January 18, 2023 04:09 — forked from ssrihari/clojure-learning-list.md
An opinionated list of excellent Clojure learning materials

An opinionated list of excellent Clojure learning materials

These resources (articles, books, and videos) are useful when you're starting to learn the language, or when you're learning a specific part of the language. This an opinionated list, no doubt. I've compiled this list from writing and teaching Clojure over the last 10 years.

  • 🔴 Mandatory (for both beginners and intermediates)
  • 🟩 For beginners
  • 🟨 For intermediates

Table of contents

  1. Getting into the language