Skip to content

Instantly share code, notes, and snippets.

View peterwwillis's full-sized avatar

Peter W peterwwillis

View GitHub Profile
@peterwwillis
peterwwillis / git-prune.sh
Created July 28, 2018 18:58
Prune Git's local orphan references, merged feature branches, remote tracking references, and more
#!/bin/bash
# Prune local orphan refs
git prune -v
# Deletes all stale (local) remote-tracking branches under origin.
# These stale branches have already been removed from the remote repository
# referenced by <name>, but are still locally available.
git remote prune origin
@peterwwillis
peterwwillis / CloudBlockStorePrices.csv
Created April 23, 2021 19:35
Comparison of cloud storage vendor prices
We can make this file beautiful and searchable if this error is corrected: It looks like row 8 should actually have 13 columns, instead of 1. in line 7.
,Linode,DigitalOcean,UpCloud,OVHCloud,Vultr,IBMCloud,Wasabi,Backblaze,AWS S3,Azure,GoogleCloud,Rackspace
Prices,https://www.linode.com/products/object-storage/,https://www.digitalocean.com/pricing/#spaces-object-storage,,https://www.ovhcloud.com/asia/public-cloud/prices/#439,https://www.vultr.com/products/object-storage/#pricing,https://cloud.ibm.com/objectstorage/create#pricing,https://wasabi.com/cloud-storage-pricing/pricing-faqs/,https://www.backblaze.com/b2/cloud-storage-pricing.html,,,,https://www.rackspace.com/openstack/public/pricing
,,,,,,,,,,,,
TrafficIncoming,included,included,,included,included,,included,,,,,
TrafficOutgoing,1TB + 0.01 per GB,1TB + 0.01 per GB,,$11 per 1 TB,1 TB + 0.01 per GB,$90 per 1 TB,included if not exceeding storage amount,$10 per 1 TB,$90 per 1 TB,$87 per 1 TB,$120 per 1 TB,$120 per 1 TB
StoragePricePerMonth,$20 per 1 TB,$20 per 1 TB,,$10 per 1 TB,$20 per 1 TB,$22.7 per 1 TB,$6 per 1 TB,$5 per 1 TB,$21 per 1 TB,$18 per 1 TB,$20 per 1 TB,$100 per 1 TB
,,,,,,,,,,,,
ArchiveTraf
@peterwwillis
peterwwillis / k8s_debugging_tips.md
Last active March 27, 2024 14:53
Kubernetes debugging tips

Log Files

Nodes

Control Plane log files

  • /var/log/kube-apiserver.log - API Server, responsible for serving the API
  • /var/log/kube-scheduler.log - Scheduler, responsible for making scheduling decisions
  • /var/log/kube-controller-manager.log - a component that runs most Kubernetes built-in controllers, with the notable exception of scheduling (the kube-scheduler handles scheduling).

Worker Node log files

@peterwwillis
peterwwillis / .gitconfig
Created January 9, 2024 08:13
Running 1Password Desktop with Docker on Alpine Linux
[user]
; NOTE: Replace your name, email, and signing ssh public key here.
name = My Git User Name Here
email = MYGITEMAIL@ADDRESS.HERE
signingkey = MYLONGSSHPUBKEYHERE
; NOTE: Uncomment this if you want to set a default credential store for Git.
; On Linux, "secretservice" is the libsecret (aka keyring) method.
;[credential]
; credentialStore = secretservice
@peterwwillis
peterwwillis / LINUX_ACTIVE_DIRECTORY_SSSD_HOWTO.md
Created July 13, 2020 12:09
How to set up an Ubuntu 18.04 Linux system to use sssd to authenticate users using Active Directory without joining a domain

Set up Ubuntu Linux to use Active Directory for user authentication + authorization

This guide will step you through setting up an Ubuntu 18.04 Linux system so that you can login to it using an Active Directory server for authentication and authorization. NOTE: You do not need to join a domain to use this method!!

The net effect of this guide is that you do not need to ever set up a user on your Linux host. Its home directory will be automatically created at log-in time, and its password is checked (along with account expiration) against the Active Directory server.

Package Management is Inherently Dumb

All packaged software is just a random person trying to guess at how to install and run some random software. The package has to declare what packages it depends on, and what it conflicts with.

The only way for a package to have the correct 'depends' and 'conflicts' is for the original software to ship with an explicit map of all its dependencies and conflicts. No software does this, in part because every Linux distribution ships different packages, and thus has different dependencies and conflicts. And so, we have to build packages by hand. A human (who isn't the software developer) has to determine the correct dependencies and conflicts (based on other packages that this human also did not create). Then they need to build the package and test it.

A package manager (dpkg) is a dumb program that does whatever you tell it to do. A package encodes its own dependencies, and the package manager fulfills the requirements as stated, or fails if it's impossible. There's n

@peterwwillis
peterwwillis / defaultenv.sh
Created May 29, 2019 14:33
Load environment variables from a file with key=value pairs and optionally execute a program
#!/bin/sh
# defaultenv.sh - Load environment variable defaults and run programs
set -e
[ x"$DEBUG" = "x1" ] && set -x
_load_envrc () {
local file="$1"; shift
if [ -r "$file" ] ; then
# Bourne shell limits how we can test and set env vars, so here I'm
@peterwwillis
peterwwillis / cliv
Last active May 2, 2021 19:01
cliv: Command-LIne wrapper to execute different Versions of binaries in different directories using different environment variables
#!/usr/bin/env sh
# cliv - Execute commands using a specific .env and directory
set -eu
_err () { printf "%s\n" "$0: Error: $*" ; exit 1 ; }
HOME="${HOME:-$(getent passwd $(id -u) | cut -d : -f 6)}"
[ -d "$HOME/.cliv" ] || mkdir -p "$HOME/.cliv"
if [ $# -lt 1 ] || [ "$1" = "-h" ] ; then
@peterwwillis
peterwwillis / gist:4d38a41b2b7bc23816af7d5f653213fa
Last active March 17, 2021 22:01
The difference between configuration formats, configuration languages, data formats, and programming languages

The difference between configuration formats, configuration languages, data formats, and programming languages

There is a lot of confusion out there about what different file formats are and how they are intended to be used. Having used a lot of them over the years, I think I can explain their differences, and when and how to use them.

Data formats

A data format is a file format for encoding data. Typically the format is structured to make it easier for machine interpreting & processing.