Skip to content

Instantly share code, notes, and snippets.

from elasticsearch import Elasticsearch, Urllib3HttpConnection
from pprint import pprint
class CompressedHttpConnection(Urllib3HttpConnection):
def __init__(self, *args, **kwargs):
super(CompressedHttpConnection, self).__init__(*args, **kwargs)
self.headers = {'Accept-Encoding': 'gzip,deflate'}
def perform_request(self, *args, **kwargs):
@bcavagnolo
bcavagnolo / install-dynamodb-local
Last active February 15, 2017 17:26
A script to install dynamodb local
#!/usr/bin/env bash
set -e
set -x
[ "${DYNAMODB_HOME}" = "" ] && DYNAMODB_HOME=/opt/dynamodb
DYNAMODB_FILE=dynamodb_local_latest.tar.gz
DYNAMODB_URL=http://dynamodb-local.s3-website-us-west-2.amazonaws.com/${DYNAMODB_FILE}
DYNAMODB_JAR=DynamoDBLocal.jar
DYNAMODB_LOG=/var/log/dynamodb.log
@bcavagnolo
bcavagnolo / seriously_skip_setUpClass
Last active October 22, 2015 18:47
seriously skip my unittest class...even the setUpClass method
import unittest
import os
FOO_IS_ON=os.environ.get('FOO')
def seriously_skip_this_if(condition, reason):
'''skip the test class (including the setUpClass method
nose (and friends) will run your setUpClass and tearDownClass methods even
@bcavagnolo
bcavagnolo / crtime.c
Last active October 27, 2015 20:26
Get creation time of an inode on linux if available
/*
* Derived from debugfs.c in e2fsprogs and inspired by:
* http://unix.stackexchange.com/questions/50177/birth-is-empty-on-ext4/50184
*
* Copyright (C) 1993 Theodore Ts'o. This file may be redistributed
* under the terms of the GNU Public License.
*
* Modifications by Robert Sanders <gt8134b@prism.gatech.edu>
*
* To build:
@bcavagnolo
bcavagnolo / asset-store-spec.md
Last active January 11, 2019 22:37
asset store spec

Asset Store Coding Challenge

The doves are loose and lots of other satellites are showing up to the party. Antennas to communicate with the ever expanding fleet are popping up like wildflowers. To keep track of them all, we need an asset store. We've already worked out the initial requirements and made a few design decisions:

  • The asset store is going to be exposed as a custom RESTful web API written in python.

  • Every asset that we store has an "asset name". There are some rules about asset names:

    • They must be globally unique across all assets
  • They can only contain alphanumeric ascii characters, underscores, and dashes

@bcavagnolo
bcavagnolo / kinesis-reader.py
Created October 26, 2017 17:59
A better kinesis consumer example in python?
# Most of the kinesis examples out there do not seem to elucidate the
# opportunities to parallelize processing of kinesis streams, nor the
# interactions of the service limits. You may want to start your
# journey by familiarizing yourself with the concepts (e.g., what is a
# shard?) and the service limits:
# http://docs.aws.amazon.com/streams/latest/dev/service-sizes-and-limits.html
# The idea is to spawn a process per shard in order to maximize
# parallelization while respecting the service limits. If you run
# multiple instances of this script (or equivalent) you will exhaust
add_ons:
cni:
disable: false
options:
calico:
felix_input_mtu: 1440
log_level: info
mode: overlay
workload_mtu: 1500
provider: calico