Skip to content

Instantly share code, notes, and snippets.

View garnaat's full-sized avatar

Mitch Garnaat garnaat

View GitHub Profile
@garnaat
garnaat / boto_mpupload_1
Created December 5, 2010 17:45
An IPython transcript showing use of S3 MultiPart Upload in boto
In [1]: import boto
In [2]: c = boto.connect_s3()
In [3]: b = c.lookup('test-1245812163')
In [4]: mp = b.initiate_multipart_upload('testmpupload2')
In [5]: mp
Out[5]: <MultiPartUpload testmpupload2>
@garnaat
garnaat / boto_mpupload_2
Created December 5, 2010 20:05
Find all current MultiPart Upload jobs for a given bucket.
In [1]: import boto
In [2]: c = boto.connect_s3()
In [3]: b = c.lookup('test-1245812163')
In [4]: for mp in b.list_multipart_uploads():
...: print mp.key_name
...:
testmpupload1
# generates the aws canonical string for the given parameters
def canonical_string(method, path, headers, expires=None,
provider=None):
if not provider:
provider = boto.provider.get_default()
interesting_headers = {}
for key in headers:
lk = key.lower()
if headers[key] != None and (lk in ['content-md5', 'content-type', 'date'] or
lk.startswith(provider.header_prefix)):
# This software code is made available "AS IS" without warranties of any
# kind. You may copy, display, modify and redistribute the software
# code either by itself or as incorporated into your code; provided that
# you do not remove any proprietary notices. Your use of this software
# code is at your own risk and you waive any claim against Amazon
# Digital Services, Inc. or its affiliates with respect to your use of
# this software code. (c) 2006 Amazon Digital Services, Inc. or its
# affiliates.
# generates the aws canonical string for the given parameters
@garnaat
garnaat / boto_config_ia.cfg
Created February 22, 2011 13:32
How to Add Internet Archive Credentials to the boto config file
[Credentials]
aws_access_key_id = <my aws access key>
aws_secret_access_key = <my aws secret key>
gs_access_key_id=<my Google Storage access key>
gs_secret_access_key=<my Google Storage secret key>
euca_access_key_id=<my Eucalyptus ECC access key>
euca_secret_access_key=<my Eucalyptus ECC secret key>
ia_access_key_id=<my Internet Archive access key>
ia_secret_access_key=<my Internet Archive secret key>
@garnaat
garnaat / access_ia_with_boto.py
Created February 22, 2011 13:38
Example showing how to access the Internet Archive with boto
import boto
# this example assumes that you have not added the AI credentials to your boto config file
ia = boto.connect_ia('<ia_access_key_id>', '<ia_secret_access_key')
stats = ia.lookup('stats')
for key in stats:
print key.name, key.size
# or, if you have already added the IA credentials to your boto file you can do this
@garnaat
garnaat / afterlaunch.py
Created May 18, 2011 12:56
Builds on https://gist.github.com/978465 by showing a few instance features
# Assumes that https://gist.github.com/978465 has already been run. Not stand-alone
# grab an Elastic IP address
address = conn.allocate_address()
# now associate with my instance
instance.use_ip(address)
# now turn on monitoring
instance.monitor()
@garnaat
garnaat / gist:1024840
Created June 14, 2011 13:01
Connect to SimpleDB in a different region
>>> import boto.sdb
>>> boto.sdb.regions()
[RegionInfo:us-east-1, RegionInfo:eu-west-1, RegionInfo:us-west-1, RegionInfo:ap-northeast-1, RegionInfo:ap-southeast-1]
>>> eu = _[1]
>>> eu
RegionInfo:eu-west-1
>>> c = eu.connect()
>>> c
SDBConnection:sdb.eu-west-1.amazonaws.com
>>> c.get_all_domains()
@garnaat
garnaat / bucket_du.py
Created September 1, 2011 13:15
Get total bytes in a bucket
# Iterates over all the keys in a bucket and totals the bytes used
# NOTE: if you have a lot of keys, this could take a LONG time.
import boto
def bucket_du(bucket_name):
s3 = boto.connect_s3()
bucket = s3.lookup(bucket_name)
total_bytes = 0
for key in bucket:
@garnaat
garnaat / boto.cfg
Created October 13, 2011 12:48
Example showing Eucalyptus credentials in boto config file
[Credentials]
...
euca_access_key_id=999999999999999999999999999999999999
euca_secret_access_key=00000000000000000000000000000000000000
...
[Boto]
...
eucalyptus_host =173.205.188.130
walrus_host = 173.205.188.130
...