Skip to content

Instantly share code, notes, and snippets.

View pirate's full-sized avatar
🗃️
Archiving all the things!

Nick Sweeting pirate

🗃️
Archiving all the things!
View GitHub Profile
@pirate
pirate / imgur_backup.sh
Created May 4, 2023 05:26
Backup all Imgur URLs found in a given set of files using ripgrep + wget. Searches all text and binary files for any imgur URLs and downloads them locally.
View imgur_backup.sh
#!/usr/bin/env bash
# imgur_backup.sh
# Backup all Imgur URLs found in a given directory using ripgrep + wget. Searches all text files, binary files, PDFs, database dumps, etc. for any imgur URLs and downloads them into a local directory.
#
# Note: make sure you apt/brew install ripgrep first, and replace grep with ggrep (brew install grep) on macOS
# Usage:
#
# $ mkdir db_dumps
# $ docker-compose exec postgres env PGPASSWORD=somepassword pg_dump -U someuser somedb > ./db_dumps/db_dump.sql
# $ bash imgur_backup.sh db_dumps
@pirate
pirate / monkey_path_python_traceback.py
Created December 16, 2021 23:08
Monkey patch the Python traceback system to modify function names, line numbers, file locations, etc.
View monkey_path_python_traceback.py
# This code is a verbatim excerpt from the Jinja2 library that I'm preserving in a Gist for posterity
# https://github.com/pallets/jinja/blob/5b498453b5898257b2287f14ef6c363799f1405a/jinja2/debug.py
# -*- coding: utf-8 -*-
"""
jinja2.debug
~~~~~~~~~~~~
Implements the debug interface for Jinja. This module does some pretty
ugly stuff with the Python traceback system in order to achieve tracebacks
@pirate
pirate / pluginization_example.py
Last active August 4, 2022 14:25
Example of how to pluginize a complex app using a hooks system
View pluginization_example.py
"""
Example of a pluginized architecture breaking up a large app
with complex behavior (ArchiveBox), into a series of steps
that plugins can hook into.
(read from the bottom to top to get a quick overview)
"""
import re
import json
@pirate
pirate / asymptotic_progress_bar.py
Last active May 31, 2023 08:43
Animated CLI progress bar that fills up over N seconds, gracefully handling delays by filling slower and slower if the task exceeds the expected time. Good for when you don't know how long a task is going to take, but you still need it to A. feel fast in the avg case, and B. not freeze in the worst case (unknown total time).
View asymptotic_progress_bar.py
#!/usr/bin/env python3
# asymptotic_progress_bar.py
# MIT License © 2021
#
# A pretty non-blocking python progress bar timer that fills up asymptotically until you stop it.
# Good for when you don't know how long a task is going to take (up to some max timeout),
# but you want it to feel fast and accurate / not stuck the whole time.
# ████████████████████ 0.9% (1/60sec)
# useful for animating e.g. file copy progress, download progress, package install progress, etc.
#
@pirate
pirate / auto_timezone_from_browser.html
Last active April 11, 2021 07:13
Automatically get the user's timezone from their browser time (for Django but works elsewhere).
View auto_timezone_from_browser.html
{% load tz %}
<html>
<head>
<script>
{% get_current_timezone as TIME_ZONE %}
window.TIME_ZONE = '{{TIME_ZONE}}' // timezone server thinks we're in
function setCookie(name, value, days) {
let expires = ""
if (days) {
@pirate
pirate / django_auto_tz.py
Created March 23, 2021 15:52
Automatic timezone detection based on the user's browser for Django
View django_auto_tz.py
# Inspiration from here: https://tutti.me/post/5453/
# Or you can use a library like:
# - https://github.com/adamcharnock/django-tz-detect (last updated 2019)
# - https://github.com/Miserlou/django-easy-timezones (last updated 2016)
# - https://github.com/jamesmfriedman/django-easytz (last updated 2015)
# Or guess TZ using visitors IP GEOIP: https://codereview.stackexchange.com/questions/161261/set-time-zone-from-cookie
# settings.py
TEMPLATES = [{
@pirate
pirate / request_logging_with_filenames.js
Created March 12, 2021 01:40
Show filename:lineno for every log message in node. Log the time, origin ip, method, and url for every express.js request.
View request_logging_with_filenames.js
// patch console.log to show filename and line number for every logged message
//
// ./src/index.js:68:9 √ Connected to Sentry
// ./src/index.js:72:9 √ Connected to Stripe
// ./src/index.js:82:9 √ Connected to Backblaze
// ./src/index.js:99:9 √ Connected to FireBase
// ./src/routes/paypal.js:68:17 Error: Request failed with status code 401
// at createError (./node_modules/axios/lib/core/createError.js:16:15)
// at settle (./node_modules/axios/lib/core/settle.js:17:12)
// at IncomingMessage.handleStreamEnd (./node_modules/axios/lib/adapters/http.js:236:11)
@pirate
pirate / check_github_usernames.py
Last active March 23, 2022 00:36
Quickly check to see if each username in a list of potential GitHub usernames is available or taken
View check_github_usernames.py
#!/usr/bin/env python3
# Usage:
# pip3 install PyGithub
#
# echo someusername | python3 check_github_usernames.py
# # or
# python3 check_github_usernames.py < list_of_usernames.txt
#
# [+] Starting to check 1212 usernames on GitHub...
# X agq (is taken)
@pirate
pirate / test_stdin_stdout_stderr.py
Last active February 19, 2021 00:10
Get all the information you could ever want about stdin, stdout, and stderr file descriptors in Python (e.g. is it a TTY, terminal, pipe, redirection, etc.)
View test_stdin_stdout_stderr.py
#!/usr/bin/env python3
# Get all the information you could ever want about the STDIN, STDOUT, STDERR file descriptors (e.g. is it a TTY, terminal, pipe, redirection, etc.)
# Works cross-platform on Windows, macOS, Linux, in Docker, and in Docker-Compose by using stat library
#
# Useful for detecting and handling different stdin/stdout redirect scenarios in CLI scripts,
# e.g. is the user piping a file in or are they interactively typing things in?
# is the process output being saved to a file or being printed to a terminal?
# can we ask the user for input or is it a non-interactive masquerading as a TTY?
#
# Further reading:
@pirate
pirate / periodicrestarter.supervisord.conf
Created February 4, 2021 10:15
Periodically restart some services every n seconds/minutes/hours with supervisord
View periodicrestarter.supervisord.conf
[program:example-server]
command=npm run start:dev
directory=/opt/%(program_name)s
stdout_logfile=/opt/%(program_name)s/data/logs/server.log
redirect_stderr=true
autostart=true
autorestart=true
startretries=5
stopwaitsecs=30
stopsignal=TERM