Skip to content

Instantly share code, notes, and snippets.

@mdaniel
mdaniel / git-svn-revmap.awk
Created March 1, 2012 21:40
regenerate git-svn .rev_map file
#! /usr/bin/env gawk -f
# git --no-pager log | gawk -f this-script > revmap.txt
# xxd -c 24 -r revmap.txt \
# .git/svn/refs/remotes/trunk/.rev_map.cfe28804-0f27-0410-a406-dd0f0b0b656f
# 0000000: 0000 0001 cce6 285e 48e1 e3cc 0d7d 0617 beb0 4e88 a126 8634
# 0000018: 0000 0006 6e4f ada4 bed4 6660 c987 dc74 1243 8973 16cc c810
BEGIN {
FS=" "
# this number comes from ``git --no-pager log | grep ^commit | wc -l``
# and is ZERO-indexed because the last xxd will begin with 0000
@mdaniel
mdaniel / Vagrantfile
Last active February 21, 2021 10:30
Provision a virtual machine for building ChromeOS using Vagrant <http://www.vagrantup.com>. Just download this into a directory, name it ``Vagrantfile`` (if it isn't already), run ``vagrant up`` followed by ``vagrant ssh`` and you'll see two shell scripts there, ready to sync up the ChromeOS source code.
# -*- mode: ruby -*-
# vi: set ft=ruby :
Vagrant.configure("2") do |config|
# All Vagrant configuration is done here. The most common configuration
# options are documented and commented below. For a complete reference,
# please see the online documentation at vagrantup.com.
# Every Vagrant virtual environment requires a box to build off of.
config.vm.box = "ubuntu1204_64"
@mdaniel
mdaniel / har_to_webscarab.py
Last active December 20, 2015 19:19
Convert the .har files saved by Chrome into a OWASP WebScarab session This is obviously an imperfect conversion since Chrome chooses to omit the response body more often than not.
import calendar
import json
import os
import re
import sys
import time
import urllib
class Conversation:
"""
@mdaniel
mdaniel / bbc_programme.py
Created September 23, 2013 06:22
Outputs a textual description of the provided BBC programme URL (or local file).
#! /usr/bin/env python
import sys
from urllib import urlopen
from bs4 import BeautifulSoup
def build_xmlns_alias_map( html_tag ):
"""
:param html_tag: the `html` Tag that contains the `xmlns` attributes
:return: a dict whose key is the namespace and whose value is the alias in this document
"""
// ==UserScript==
// @name Relocate the Close button away from its Comment friend
// @namespace http://greasemonkey.matthewdaniel.com/
// @version 0.2
// @description Just like the name says
// @match https://github.com/*/pull/*
// @include https://github.com/*/pull/*
// @match http://github.com/*/pull/*
// @include http://github.com/*/pull/*
// ==/UserScript==
@mdaniel
mdaniel / Vagrantfile.sh
Last active March 26, 2017 23:57
Vagrantfile.sh script to provision an EMR compatible (hadoop 1.0.3, JavaSE 1.6) Vagrant instance
#! /bin/sh
set -e
set -x
## this script is run as `sudo /tmp/vagrant-shell` by ~vagrant
#echo '[USER]'
#id -a
#echo '[ENV]'
#env
@mdaniel
mdaniel / logging_mdc.py
Created January 10, 2014 05:47
Just like the docstring says: Demonstrates how a [Mapped Diagnostic Context](http://logback.qos.ch/manual/mdc.html) of log4j MDC fame (and also provided in slf4j) could be implemented in Python
# coding=utf-8
"""
Demonstrates how a [Mapped Diagnostic Context](http://logback.qos.ch/manual/mdc.html)
of log4j MDC fame (and also provided in slf4j) could be implemented in Python
"""
import json
import logging
import sys
import threading
@mdaniel
mdaniel / bug.sh
Created February 4, 2014 05:01
Reproducible reposurgeon 3.2 unicode path bug
#! /bin/bash
set -e
mkdir orig
cd orig
git init
git fast-import <../original.fi
cd ..
echo -ne "read orig\nexpunge /.jar/\nwrite >bad.fi\n" | reposurgeon
mkdir bad
cd bad
@mdaniel
mdaniel / microdata.py
Last active August 29, 2015 13:56
Dump discovered microdata descriptors (optionally along with their values) found in an HTML document
#! /usr/bin/env python
# -*- coding: utf-8 -*-
from __future__ import print_function
import sys
from bs4 import BeautifulSoup
def show_em_all(soup, with_values):
# todo: some way of marking a nested itemtype as seen?
# maybe the upward tree traversal would yield a unique path
@mdaniel
mdaniel / bs4visualizer.py
Created February 17, 2014 06:41
Use PyQt to visualize what BeautifulSoup4 sees when it ingests the html; the jsoup flavor of this actually allows one to run queries, so hopefully this one will, too (any minute now)
#! /usr/bin/env python
# -*- coding: utf-8 -*-
__docformat__ = 'reStructuredText'
import sys
from bs4 import BeautifulSoup, NavigableString, Tag
from PyQt5.Qt import (
QApplication, QMainWindow, QMenuBar, QMenu, QTreeWidget, QTreeWidgetItem)