Skip to content

Instantly share code, notes, and snippets.

View notmyname's full-sized avatar

John Dickinson notmyname

View GitHub Profile
@notmyname
notmyname / s3_delete.py
Created November 15, 2010 16:41
quickly delete keys in s3
#!/usr/bin/env python
import eventlet
eventlet.monkey_patch()
import boto
import sys
import Queue
conn = boto.connect_s3('s3 creds', 'go here')
import cloudfiles
import cf_auth
conn = cloudfiles.get_connection(username=cf_auth.username,
api_key=cf_auth.apikey)
container = conn.create_container('foo')
@notmyname
notmyname / gist:1039665
Created June 22, 2011 07:54
md5 a large file
import sys
from hashlib import md5
target = sys.argv[1]
csize = 4096
md5sum = md5()
with open(target, 'rb') as f:
blocks = iter(lambda: f.read(csize), '')
for block in blocks:
md5sum.update(block)
@notmyname
notmyname / gist:1052036
Created June 28, 2011 19:58
project autonomy
My hope for project autonomy:
- Projects are highly autonomous (code hosting, issue tracking, versioning, release schedules, branching models, developer workflow, etc)
- Projects are required to integrate with Openstack releases and support the open goals of Openstack
This implies that Openstack is a collection of affiliated projects that work well together and are distributed together as an official Openstack release. Openstack can require that projects follow good coding practices (reviews, tests, version control, etc), but these requirments should be descriptive rather than prescriptive. For example, Openstack can require that a project use version control, recommend bzr, but allow the project to choose what they want.
As an example from swift (because that's my experience), the swift devs choose their own code hosting and issue tracking. For releases, the swift team (or PTL) provides the openstack packagers info on what's changed. This can be in the form of changelogs, copies of bug reports in LP, o
DEBUG:root:git rev-parse --git-dir
DEBUG:root:git symbolic-ref HEAD
DEBUG:root:git config bzr.test_change.bzr
DEBUG:root:bzr branch /Users/john/Documents/openswift/.git/bzr/repo/master /Users/john/Documents/openswift/.git/bzr/repo/test_change
Branched 335 revision(s).
DEBUG:root:rewrite bzr marks
DEBUG:root:- :344 john.dickinson@rackspace.com-20101004214159-g6jjobzrmzv4ueiw
DEBUG:root:+ :344 john.dickinson@rackspace.com-20101004214159-g6jjobzrmzv4ueiw
DEBUG:root:- :345 john.dickinson@rackspace.com-20101004214303-5r75syunno9e918i
DEBUG:root:+ :345 john.dickinson@rackspace.com-20101004214303-5r75syunno9e918i
$ git bzr push
13:19:13 Calculating the revisions to include ...m source
13:19:13 Starting export of 1527 revisions ...
13:19:13 Exported 1527 revisions in 0:00:00
13:19:14 Starting import ...
bzr: ERROR: exceptions.KeyError: '1531'
Traceback (most recent call last):
File "/Library/Python/2.7/site-packages/bzrlib/commands.py", line 926, in exception_to_return_code
return the_callable(*args, **kwargs)
@notmyname
notmyname / swift_object_versioning.markdown
Created December 28, 2011 16:55
object versioning in swift

Object Versioning in Swift

Set X-Object-Versions header on an object. The header behaves like the X-Object-Manifest header in that it references a container/prefix of other objects in the account. When requested, the "version manifest" will perform a listing based on the value of the x-object-versions header and return the most recent version (the last object when the names are lexigraphically sorted). Likewise a PUT to the version maniest will create a new object name (derived from the x-object-versions header value and the current timestamp) and

@notmyname
notmyname / git-largest
Created January 6, 2012 19:50
git-largest command to find the largest objects in a repo's history
#!/bin/sh
COUNT=10
if [ $# -gt 0 ]; then
COUNT=$1
fi
if [ ! -r .git/objects/pack/*.idx ]; then
git gc
fi
@notmyname
notmyname / gist:2018680
Created March 11, 2012 23:32
evil with the import statement
'''evil with the import statement'''
import sys
def foo_func(x):
return x * x
sys.modules['foobar'] = foo_func
#!/usr/bin/env python
import sys
from swift.common.memcached import MemcacheRing
memcache_servers = '127.0.0.1:11211'
memcache = MemcacheRing([s.strip() for s in memcache_servers.split(',') if s.strip()])