Skip to content

Instantly share code, notes, and snippets.

@wlonkly
wlonkly / debugging.md
Last active July 8, 2021 15:13
Steps I took to troubleshoot a full disk

I wrote this down after I responded to a page today (a holiday) because it would've been a decent pairing opportunity for a couple of new people on my team. Second best is that people can read what I did afterwards and ask me any questions. And then I realized that there's nothing PagerDuty-specific or confidential in here, so I may as well share it wider. It's hardly an epic incident, but it's a good example of "doing the work", I think. I borrowed the "write down what you learned" approach from Julia "b0rk" Evans. It's a fantastic practice.

The PagerDuty incident: "Disk will be full in 12 hours. device:/dev/nvme0n1p1, host:stg-nomadusw2-client-..."

(Note for non-PD readers: We run Nomad where others might run Kubernetes.)

Here's the process I went through.

  • Noticed that the usual docker system prune -a -f didn't resolve it
  • Tried docker system prune -a -f and it cleared up 0B
# Reconstructed via infocmp from file: /usr/share/terminfo/t/tmux-256color
tmux-256color|tmux with 256 colors,
OTbs, OTpt, am, hs, km, mir, msgr, xenl, AX, G0,
colors#256, cols#80, it#8, lines#24, pairs#32767, U8#1,
acsc=++\,\,--..00``aaffgghhiijjkkllmmnnooppqqrrssttuuvvwwxxyyzz{{||}}~~,
bel=^G, blink=\E[5m, bold=\E[1m, cbt=\E[Z, civis=\E[?25l,
clear=\E[H\E[J, cnorm=\E[34h\E[?25h, cr=^M,
csr=\E[%i%p1%d;%p2%dr, cub=\E[%p1%dD, cub1=^H,
cud=\E[%p1%dB, cud1=^J, cuf=\E[%p1%dC, cuf1=\E[C,
cup=\E[%i%p1%d;%p2%dH, cuu=\E[%p1%dA, cuu1=\EM,
@jj1bdx
jj1bdx / search_devicename_example.sh
Created July 23, 2019 04:10
Using macOS diskutil output to search the device identifier for a given volume name
#!/bin/sh
# search mount point based on the device name
search_devicename () {
diskutil list -plist | \
plutil -convert json -r -o - - | \
jq -c '[.AllDisksAndPartitions[].APFSVolumes, .AllDisksAndPartitions[].Partitions] | flatten | map(select(has("VolumeName")))[] | {volume: .VolumeName?, id: .DeviceIdentifier?}' | \
jq -r --arg volname "$1" 'select(.volume == $volname) | .id?'
}
@aparrish
aparrish / spacy_intro.ipynb
Last active August 9, 2023 01:41
NLP Concepts with spaCy. Code examples released under CC0 https://creativecommons.org/choose/zero/, other text released under CC BY 4.0 https://creativecommons.org/licenses/by/4.0/
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

NIPS ML in the Law symposium 2016 notes

Including notes for the second session and the first panel.

My three takeaways

  • Tech folks have a tendency to "fly in and fix everything." That feels like a dangerous approach here. It's far better to stand on the shoulders of existing legal precedent, which has studied fairness, discrimination, and bias for decades, even if that slows down progress.
  • Machine learning systems mirror and amplify bias by default. We cannot simply ignore sensitive attributes because the system averages loss over the majority. (Disparate mistreatment). Pithy corollary: this problem will only go away if we devote resources into making it go away.
  • Providing explanations for decisions is the only humane way to build automatic classification systems. Why? If I can't test a result, I can't contest it. If the decisions must be testable and explainable, they will be much more reliable as a result.

Aaron Roth: Quantitative tradeoffs between fairness and accuracy in machine learning

@crlane
crlane / trie.py
Last active March 17, 2024 15:54
An implementation of a trie in python using defaultdict and recursion
from collections import defaultdict
def node():
return defaultdict(node)
def word_exists(word, node):
if not word:
return None in node
return word_exists(word[1:], node[word[0]])
@karpathy
karpathy / min-char-rnn.py
Last active July 22, 2024 04:44
Minimal character-level language model with a Vanilla Recurrent Neural Network, in Python/numpy
"""
Minimal character-level Vanilla RNN model. Written by Andrej Karpathy (@karpathy)
BSD License
"""
import numpy as np
# data I/O
data = open('input.txt', 'r').read() # should be simple plain text file
chars = list(set(data))
data_size, vocab_size = len(data), len(chars)
@chrismdp
chrismdp / s3.sh
Last active March 5, 2024 12:57
Uploading to S3 in 18 lines of Shell (used to upload builds for http://soltrader.net)
# You don't need Fog in Ruby or some other library to upload to S3 -- shell works perfectly fine
# This is how I upload my new Sol Trader builds (http://soltrader.net)
# Based on a modified script from here: http://tmont.com/blargh/2014/1/uploading-to-s3-in-bash
S3KEY="my aws key"
S3SECRET="my aws secret" # pass these in
function putS3
{
path=$1
@burke
burke / remotepaste.md
Last active June 15, 2024 12:55
This sets up keybindings in tmux that allow you to copy/paste to/from your OS X clipboard from tmux running inside an SSH connection to a remote host. Partially borrowed from http://seancoates.com/blogs/remote-pbcopy

Local (OS X) Side

~/Library/LaunchAgents/pbcopy.plist

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
     <key>Label</key>
     <string>localhost.pbcopy</string>
@darcyparker
darcyparker / vimModeStateDiagram.svg
Last active July 4, 2024 12:03
Vim Modes Transition Diagram in SVG https://rawgithub.com/darcyparker/1886716/raw/eab57dfe784f016085251771d65a75a471ca22d4/vimModeStateDiagram.svg Note, most of the nodes in this graph have clickable hyperlinks to documentation.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.