xtao

# It's important to convert the vbox image (VMDK or VDI or whatever) using
# the same version of VirtualBox that created it. You can try converting the image
# with qemu-img or kvm-img, but weird version mismatches will possibly make it not
# work.

# On your VirtualBox machine:

cd $VBOX_ROOT/$MACHINE_ROOT/
VBoxManage clonehd machine.vmdk machine.img --format RAW
scp machine.img root@kvm-host:/somewhere

xtao

# This document describes updating the SmartOS boot_archive and /usr volumes.
#
# The boot_archive effectively contains /. If you want to make changes to SMF
# or so forth, you need to update the boot_archive.
#
# If you want to add things to /usr, such as drivers, you will need to update
# usr.lgz. It is compressed, so note the lofiadm -U and -C calls below. Once
# you have updated usr.lgz you will need to copy it back to the mounted
# boot_arhive.
#

xtao

So, to get something like /etc/rc.local you can use the custom SMF import facility. (See the source for more information about how this actually works.)

/opt is mounted out of zones/opt by default. You can create a directory /opt/custom/smf and populate it with SMF manifests. Any manifests you put in there will be imported by SmartOS when it boots. Below is an example SMF manifest that simply starts /opt/custom/bin/postboot, a self-explanatory shell script that you can use like /etc/rc.local.

Note that it would likely be better to customise and respin your own images, as putting a bunch of platform state in the zones pool undoes some of the benefits of the ramdisk platform architecture that SmartOS has.

xtao

Virtualenv's bin/activate is Doing It Wrong

I'm a Python programmer and frequently work with the excellent [virtualenv][] tool by Ian Bicking.

Virtualenv is a great tool on the whole but there is one glaring problem: the activate script that virtualenv provides as a convenience to enable its functionality requires you to source it with your shell to invoke it. The activate script sets some environment variables in your current environment and defines for you a deactivate shell function which will (attempt to) help you to undo those changes later.

This pattern is abhorrently wrong and un-unix-y. activate should instead do what ssh-agent does, and launch a sub-shell or sub-command with a modified environment.

Problems

xtao

Applies to SmartOS/illumOS/Solaris machines with the mkfile command.

For testing too many file effects, create 500,000 numbered 2k zero padded files:
#./manyfiles.sh 500000 2k


For maxing out file storage, then releasing, this makes 20 100Gb zero padded files, bzips them. 
# ./manyfiles.sh 20 100g
# pbzip2 TEMP_* &
The 20 resulting files should be all exactly the same size in bytes.

xtao

# Optimized for writes, sort on read

# LVC
redis.hset("bonds|1", "bid_price", 96.01)
redis.hset("bonds|1", "ask_price", 97.53)
redis.hset("bonds|2", "bid_price", 95.50)
redis.hset("bonds|2", "ask_price", 98.25)
redis.sadd("bond_ids", 1)
redis.sadd("bond_ids", 2)

xtao

Test Gist Dirs.

xtao

#!/usr/bin/env bash

# Copyright (c) 2012, Richard Yao ryao@cs.stonybrook.edu

# Permission to use, copy, modify, and/or distribute this software for any
# purpose with or without fee is hereby granted, provided that the above
# copyright notice and this permission notice appear in all copies.

# THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH
# REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY

xtao

Architectural Questions

• Are responsibilities well defined?
• Are the collaborations well defined?
• Is coupling minimized?
• Can you identify potential duplication?
• Are interface definitions and constraints acceptable?
• Can modules access needed data—when needed?

xtao

Latency numbers every programmer should know

L1 cache reference ......................... 0.5 ns
Branch mispredict ............................ 5 ns
L2 cache reference ........................... 7 ns
Mutex lock/unlock ........................... 25 ns
Main memory reference ...................... 100 ns             
Compress 1K bytes with Zippy ............. 3,000 ns  =   3 µs
Send 2K bytes over 1 Gbps network ....... 20,000 ns  =  20 µs
SSD random read ........................ 150,000 ns  = 150 µs

Read 1 MB sequentially from memory ..... 250,000 ns = 250 µs