Skip to content

Instantly share code, notes, and snippets.

View apergos's full-sized avatar
💭
Hanging my dirty l̶a̶u̶n̶d̶r̶y̶ code out in public

Ariel T. Glenn apergos

💭
Hanging my dirty l̶a̶u̶n̶d̶r̶y̶ code out in public
View GitHub Profile
@apergos
apergos / emergency_refsaver.py
Created January 28, 2017 14:32
Read urls from a file, preserve their contents at archive.org, log the original and archive urls. Edit to add your contact email in user agent and remove the warning/exit.
#! /usr/bin/env python
import sys
import getopt
import requests
def usage(message=None):
'display a usage message'
if message:
sys.stderr.write(message + "\n")
import getopt
import sys
import json
import time
import types
import requests
# pylint: disable=broad-except
#!/usr/bin/python3
'''
recombine the multistream index file into an output
file path of your choice, for a given wiki and date
'''
import os
import re
import bz2
import getopt
@apergos
apergos / generate_glyph_pngs.py
Last active August 23, 2019 09:55
generate a png with a single character glyph, border and background
#!/usr/bin/python3
'''
to be used to generate a pile of images for import to deployment-prep commons
requires: pycairo, Pillow >= 6.0.0
'''
import sys
import cairo
from PIL import Image
from PIL.PngImagePlugin import PngInfo
#!/bin/bash
# for each host in the list, display the dump type
# requires sudo root on snapshot hosts (dumpsgen is not enough)
# fixme generate this list from someplace. it would be nicer.
hostnums="05 06 07 08 09 10"
get_dumptype() {
host="$1"
#!/bin/bash
usage() {
echo "Show which db hosts in which dc are in some group, for a given section"
echo
echo "Usage:"
echo "$0 section_number group"
echo
echo " section_number should be one of s1,s2,... or DEFAULT"
echo " this argument is required"
#!/usr/bin/python3
'''
sample methods illustrating the use and innards of some
classes from the command_management module
'''
import fcntl
import select
import sys
import os
from dumps.commandmanagement import CommandPipeline, CommandSeries
@apergos
apergos / wikidump.conf.dumps
Created December 18, 2020 13:18
wikidump.conf.dumps as of December 2020 on Wikimedia snapshot hosts
#############################################################
# This file is maintained by puppet!
# modules/snapshot/templates/dumps/wikidump.conf.erb
#############################################################
[wiki]
dblist=/srv/mediawiki/dblists/all.dblist
privatelist=/srv/mediawiki/dblists/private.dblist
closedlist=/srv/mediawiki/dblists/closed.dblist
skipdblist=/etc/dumps/dblists/skip.dblist
@apergos
apergos / stages_partial
Created December 18, 2020 13:23
stages file for partial dump runs of regular and big wikis, from December 2020
#############################################################
# This file is maintained by puppet!
# modules/snapshot/templates/dumps/dumpstages.erb
#############################################################
# run these commands in order on the regular snapshot hosts
# slots_used numcommands on_failure error_notify command
@apergos
apergos / show_ubns.py
Last active February 6, 2021 17:38
Display basic info about recent unbreak-now tasks from Phabricator projects
#!/usr/bin/python3
'''
I needed a script to show me all the ubns in the last few days on a couple
projects, on demand. So, here it is.
'''
import getopt
import os
import sys
import json
import time