Skip to content

Instantly share code, notes, and snippets.

View miketheman's full-sized avatar

Mike Fiedler miketheman

View GitHub Profile
@miketheman
miketheman / zook_grow.md
Created July 22, 2013 21:36
Adding nodes to a ZooKeeper ensemble

Adding 2 nodes to an existing 3-node ZooKeeper ensemble without losing the Quorum

Since many deployments may start out with 3 nodes and so little is known about how to grow a cluster from 3 memebrs to 5 members without losing the existing Quorum, here is an example of how this might be achieved.

In this example, all 5 nodes will be running on the same Vagrant host for the purpose of illustration, running on distinct configurations (ports and data directories) without the actual load of clients.

YMMV. Caveat usufructuarius.

Step 1: Have a healthy 3-node ensemble

# Checkout warehouse at the current `HEAD` for reproducability
git checkout aa5b2ec11745299dacd60cb7f73a066723372a9e
# Need to apply some changes to the core to allow un-hashed (unreleased) versions, and building from GitHub source.
# Snag the diff file and apply it
curl https://gist.githubusercontent.com/miketheman/178429ea5ae4f7ff2b360ac85cd9e1f2/raw/05c7a98464b41e224bf27f85f9a8b903669e6fb7/test-coverage.diff -o tes
t-coverage.diff
patch < test-coverage.diff
@miketheman
miketheman / query.sql
Created August 24, 2024 12:25
PyPI Download stats, past 7 days, grouped by whether user-agent self-reports `ci: true`
SELECT
DATE(timestamp) AS download_date,
COUNTIF(details.ci = TRUE) AS ci_downloads,
COUNTIF(details.ci IS NULL OR details.ci = FALSE) AS non_ci_downloads
FROM `bigquery-public-data.pypi.file_downloads`
WHERE DATE(timestamp) >= DATE_SUB(CURRENT_DATE(), INTERVAL 7 DAY)
GROUP BY download_date
ORDER BY download_date;
@miketheman
miketheman / security-group-cleanup.py
Created May 7, 2012 21:07
AWS EC2 Unused Security Group cleanup
#!/usr/bin/env python
import sys
import boto
import pprint
del_flag = ''
if len(sys.argv) > 1:
del_flag = sys.argv[1]
from unittest.mock import patch
import boto3
from botocore.stub import Stubber
# create an ssm client
ssm = boto3.client("ssm")
# create an sqs client
sqs = boto3.client("sqs")
@miketheman
miketheman / main.py
Last active December 22, 2022 19:20
Add a bunch of AWS Heroes (and some other folks, too) to a Mastodon list.
"""
Add users to a Mastodon instance list.
Uses https://mastodonpy.readthedocs.io/
"""
from os import environ
from pathlib import Path
from mastodon import Mastodon
from mastodon.errors import MastodonNotFoundError, MastodonServiceUnavailableError

Tests how GitHub renders code blocks

Code fence, no language set

    ```
    echo "foo"
    ```
@miketheman
miketheman / Makefile
Created September 1, 2016 19:54
Django + docker-compose Makefile
CURRENT_DIRECTORY := $(shell pwd)
TESTSCOPE = apps
TESTFLAGS = --with-timer --timer-top-n 10 --keepdb
help:
@echo "Docker Compose Help"
@echo "-----------------------"
@echo ""
@echo "Run tests to ensure current state is good:"
@miketheman
miketheman / Makefile
Created November 22, 2021 17:32
Makefile with targets to generate static renderings from PlantUML sources
DIAGRAMS := $(wildcard *.puml */*puml)
SVGS := $(DIAGRAMS:.puml=.svg)
PNGS := $(DIAGRAMS:.puml=.png)
all: $(SVGS) $(PNGS)
%.png: %.puml
plantuml -tpng "$<"
%.svg: %.puml
@miketheman
miketheman / enable_alerting.py
Created February 14, 2020 14:39
A simple script to enable GitHub's Data Services & Vulnerability Scanning on a given Organization's Repositories
from github import Github
gh = Github('SET_GITHUB_ACCESS_TOKE_HERE')
org = gh.get_organization('SET_ORG_NAME_HERE')
repos = org.get_repos()
for repo in repos:
if repo.archived: