Skip to content

Instantly share code, notes, and snippets.

View cemeyer's full-sized avatar

Conrad Meyer cemeyer

  • Seattle, Washington
View GitHub Profile
@cemeyer
cemeyer / 2013-09-15.md
Created September 15, 2013 16:02
Hack: Improving XML bad-character filtering performance by 10x (Qt4)

Backstory: Some [Unicode code points are invalid in XML][invalid]; specifically, we want to filter the code points (hex) 0-8, b-c, and e-1f. Qt4's QXmlStreamWriter() makes no attempt to filter these codepoints in the documents it writes.

So, here's our baseline, with the same benchmark we'll use later (big.gpx is a giant 142 MB file, and GPX is an XML-based format):

./gpsbabel -i gpx -f big.gpx [-o gpx -F /dev/null] (x5)

17.2 user, 4.8 system, 22.1 total

In [GPSBabel][], we already subclass QXmlStreamWriter to add other convenience methods. So our initial attempt to filter bad characters was simply to interpose on all methods that take potentially unfiltered inputs (writeAttribute(), writeTextElement(), ...) and apply a QString::replace() substitution with a QRegExp.

@cemeyer
cemeyer / stream_errorhandling_hack.py
Created December 2, 2018 00:41
[Unix] Hack: Change the error-handling on a file-like stream in Python3 (e.g., sys.stdout)
import os
import sys
# Re-opens the file-like stream of string 'attr' in parent object 'pobj' with
# the open() builtin parameters 'args' and 'kwargs'. Unix-specific.
#
# I would love it if there was a better way to do this and I just missed it.
def set_stream_errorh(pobj, attr, *args, **kwargs):
obj = getattr(pobj, attr)
fileno = obj.fileno()
@cemeyer
cemeyer / setx.py
Created January 13, 2019 00:38
Python2.7 'set -x' alike
# Use like: python setx.py my_python_program [args...]
#
# my_python_program can be a literal path to a file (e.g., foo.py), or the unqualified
# name of a Python program in PATH. In the latter case, setx.py examines PATH for
# a program with that name.
#
# Prints every line executed, including the filename and line number.
#
# This is a hacky prototype for py27; it almost certainly breaks in significant ways on
# non-Unix, or Python3000; and has only been tested on a FreeBSD derivative. It still
@cemeyer
cemeyer / nailgun.py
Created January 18, 2019 02:20
Poor man's nailgun, for when some imports are stupid slow
# Other code goes here
def run():
pass
# ...
def main():
global nailgun
while True:
try:
@cemeyer
cemeyer / Migrate VMware workstation to qemu+kvm
Last active June 27, 2019 23:18
Steps to migrate a VMware VM to KVM
1) Deleted all vmware snapshots for the VM. sync. sync.
2) Followed rwmj's guide to setting up a br0 device for libvirt in Fedora.[0]
3) Grabbed vmware2libvirt[1] from Ubuntu (it's just a standalone python script) and
used it with --bridge br0 -f foo.vmx > foo.xml.
4) Converted my vmdk to raw with qemu-img[2], moved it to /var/lib/libvirtd per
instructions[3]
5) Edited my xml to point to the right disk, use the qemu/raw driver, use the right
path for kvm (/usr/bin/qemu-kvm on Fedora).
5.a) Configure a unix socket serial device (like VMware's "pipe" mode):
@cemeyer
cemeyer / ⁄etc⁄logrotate.conf
Last active October 22, 2023 12:06
Sane logrotate.conf settings
# create new (empty) log files after rotating old ones
create
# use date as a suffix of the rotated file
dateext
compress
compresscmd /bin/zstd
compressext .zst
compressoptions -18 -T0 --rm