Skip to content

Instantly share code, notes, and snippets.

View apergos's full-sized avatar
💭
Hanging my dirty l̶a̶u̶n̶d̶r̶y̶ code out in public

Ariel T. Glenn apergos

💭
Hanging my dirty l̶a̶u̶n̶d̶r̶y̶ code out in public
View GitHub Profile
#!/usr/bin/python3
# encoding: utf-8
'''
download Wikimedia Enterprise HTML dumps for all or for specified wiki projects
'''
import os
import getopt
import logging
import sys
import time
@apergos
apergos / generate_rev_data.py
Created June 8, 2021 09:13
generate table of revision ids and times for plotting wiki project growth
#!/usr/bin/python3
'''
generate a table of rev id number vs time in 1 million revision increments,
suitable for feeding to gnuplot to chart project growth.
'''
import getopt
@apergos
apergos / cast_edit.py
Created May 24, 2021 05:01
convert an asciicast file to a format nicer for hand editing, and back as well
#!/usr/bin/python3
'''
the most wordy python script ever. this could probably be done in 2 lines
of awek but I was too lazy to look up the details.
convert an asciicast file ([timestamp, type, text-string]) to one
with [time-since-last-entry, type, text-string] entries or vice versa,
so that an asciicast file can be converted to interval format, lines
removed that are mistakes or unwanted, and then converted back
@apergos
apergos / show_ubns.py
Last active February 6, 2021 17:38
Display basic info about recent unbreak-now tasks from Phabricator projects
#!/usr/bin/python3
'''
I needed a script to show me all the ubns in the last few days on a couple
projects, on demand. So, here it is.
'''
import getopt
import os
import sys
import json
import time
@apergos
apergos / stages_partial
Created December 18, 2020 13:23
stages file for partial dump runs of regular and big wikis, from December 2020
#############################################################
# This file is maintained by puppet!
# modules/snapshot/templates/dumps/dumpstages.erb
#############################################################
# run these commands in order on the regular snapshot hosts
# slots_used numcommands on_failure error_notify command
@apergos
apergos / wikidump.conf.dumps
Created December 18, 2020 13:18
wikidump.conf.dumps as of December 2020 on Wikimedia snapshot hosts
#############################################################
# This file is maintained by puppet!
# modules/snapshot/templates/dumps/wikidump.conf.erb
#############################################################
[wiki]
dblist=/srv/mediawiki/dblists/all.dblist
privatelist=/srv/mediawiki/dblists/private.dblist
closedlist=/srv/mediawiki/dblists/closed.dblist
skipdblist=/etc/dumps/dblists/skip.dblist
#!/usr/bin/python3
'''
sample methods illustrating the use and innards of some
classes from the command_management module
'''
import fcntl
import select
import sys
import os
from dumps.commandmanagement import CommandPipeline, CommandSeries
#!/bin/bash
usage() {
echo "Show which db hosts in which dc are in some group, for a given section"
echo
echo "Usage:"
echo "$0 section_number group"
echo
echo " section_number should be one of s1,s2,... or DEFAULT"
echo " this argument is required"
#!/bin/bash
# for each host in the list, display the dump type
# requires sudo root on snapshot hosts (dumpsgen is not enough)
# fixme generate this list from someplace. it would be nicer.
hostnums="05 06 07 08 09 10"
get_dumptype() {
host="$1"
@apergos
apergos / generate_glyph_pngs.py
Last active August 23, 2019 09:55
generate a png with a single character glyph, border and background
#!/usr/bin/python3
'''
to be used to generate a pile of images for import to deployment-prep commons
requires: pycairo, Pillow >= 6.0.0
'''
import sys
import cairo
from PIL import Image
from PIL.PngImagePlugin import PngInfo