Skip to content

Instantly share code, notes, and snippets.

View NostraDavid's full-sized avatar
🏠
Working from home

NostraDavid NostraDavid

🏠
Working from home
View GitHub Profile
@NostraDavid
NostraDavid / process_better.py
Created November 19, 2024 17:00
Polars > Spark, for small (sub 200GB) datasets
# /// script
# requires-python = ">=3.13"
# dependencies = [
# "polars",
# ]
# ///
from pathlib import Path
from typing import OrderedDict
import polars as pl
@NostraDavid
NostraDavid / .wezterm.lua
Created October 5, 2024 10:21
Default Windows wezterm.lua
-- Pull in the wezterm API
local wezterm = require 'wezterm'
-- This will hold the configuration.
local config = wezterm.config_builder()
-- Set font to JetBrains Mono NF
config.font = wezterm.font('JetBrainsMono Nerd Font')
config.font_size = 10
@NostraDavid
NostraDavid / pde.md
Created September 19, 2024 09:56
Pythonic Data Engineer (GitHub)

Pythonic Data Engineering Style Guide

The Essence of Style

Our coding style is a collective effort, blending engineering and art, aiming for precision and creativity. The best is yet to come.

Why Have Style?

Good style advances our design goals: readability, maintainability, performance,

@NostraDavid
NostraDavid / get_repos.sh
Created September 18, 2024 17:40
Grab all your repos from github and put them in subfolders per user, using curl, jq and git.
#!/usr/bin/env bash
# Set your GitHub token here
GITHUB_TOKEN="<token goes here>"
# Make the cURL request to GitHub API and store the response
response=$(curl -s -H "Authorization: token $GITHUB_TOKEN" \
-H "Accept: application/vnd.github+json" \
"https://api.github.com/user/repos?per_page=100")
@NostraDavid
NostraDavid / debian_12_setup.sh
Created June 21, 2023 19:26
Debian 12 setup
# in Terminal
sudo --login
# as su
usermod -aG sudo david
# ctrl-d
# reboot
# back in terminal
sudo apt install flatpak
sudo flatpak remote-add --if-not-exists flathub https://flathub.org/repo/flathub.flatpakrepo
@NostraDavid
NostraDavid / export_chatgpt.js
Created April 3, 2023 19:44
Save this as a bookmark. Go to a ChatGPT chat. Click the bookmark to export the file, with the UUID and Chat Title in the filename
javascript: (() => {let output = "";const es = document.querySelectorAll("[class*='min-h-[20px]']");const today = `${new Date().toISOString().slice(0, 19)}Z`;output += `---\ndate: "${today}"\n---\n#\n%60;for (let i = 0; i < es.length; i++) {const first = es[i].firstChild;if (!first) continue;if (first.nodeType === Node.TEXT_NODE) {let lines = first.textContent.split("\n"); for (let i = 0; i < lines.length; i++) {const line = lines[i];output += %60\n> ${line}%60;}output += %60\n%60;}if (first.nodeType === Node.ELEMENT_NODE) {const childNodes = first.childNodes;for (let j = 0; j < childNodes.length; j++) {const childNode = childNodes[j];if (childNode.nodeType === Node.ELEMENT_NODE) {let tagName = childNode.tagName;let textContent = childNode.textContent;switch (tagName) {case "P":output += %60\n${textContent}%60;break;case "OL":output += %60\n%60;childNode.childNodes.forEach((el, index) => {if (el.nodeType === Node.ELEMENT_NODE && el.tagName === "LI") {output += %60${index + 1}. ${el.textContent}\n%60;}});break
# Note that the "# %%" thing is from Jupyter Notebooks, which should be supported by vscode by default :)
# Just run the code with the "run code" button that pops up right above the "# %%"
# %%
import functools
from typing import Callable
ComposableFunction = Callable[[float], float]
def compose(*functions: ComposableFunction) -> ComposableFunction:
@NostraDavid
NostraDavid / .gitignore
Last active September 18, 2024 17:40
Whitelisting gitignore, instead of the default blacklisting.
# ignore all root items, not recursively
/*
# whitelist dirs
!docs
!example_data
!requirements
!src
!tests
@NostraDavid
NostraDavid / pre-commit-config.yaml
Created December 22, 2021 13:32
A basic pre-commit config
---
repos:
# update the rev versions with a `pre-commit autoupdate`
- repo: https://github.com/asottile/pyupgrade
rev: v2.29.1
hooks:
- id: pyupgrade
args: [--py39-plus]
- repo: https://github.com/myint/autoflake
rev: v1.4
@NostraDavid
NostraDavid / warcraft-credits.csv
Created July 24, 2021 21:09
Table of Warcraft 1,2,3,TFT and Vanilla WoW Credits
We can make this file beautiful and searchable if this error is corrected: No commas found in this CSV file in line 0.
Insert Order Warcraft version Subtitle Role Person
1 1 Orcs & Humans Executive Producer Allen Adham
2 1 Orcs & Humans Producer Patrick Wyatt
3 1 Orcs & Humans Programmer Patrick Wyatt
4 1 Orcs & Humans Programmer Bob Fitch
5 1 Orcs & Humans Programmer Mike Morhaime
6 1 Orcs & Humans Programmer Jesse McReynolds
7 1 Orcs & Humans Artist Stuart Rose
8 1 Orcs & Humans Artist Samwise Didier
9 1 Orcs & Humans Artist Ron Millar