skydan

Currently, there is an explosion of tools that aim to manage secrets for automated, cloud native infrastructure management. Daniel Somerfield did some work classifying the various approaches, but (as far as I know) no one has made a recent effort to summarize the various tools.

This is an attempt to give a quick overview of what can be found out there. The list is alphabetical. There will be tools that are missing, and some of the facts might be wrong--I welcome your corrections. For the purpose, I can be reached via @maxvt on Twitter, or just leave me a comment here.

There is a companion feature matrix of various tools. Comments are welcome in the same manner.

skydan

Scaling your API with rate limiters

The following are examples of the four types rate limiters discussed in the accompanying blog post. In the examples below I've used pseudocode-like Ruby, so if you're unfamiliar with Ruby you should be able to easily translate this approach to other languages. Complete examples in Ruby are also provided later in this gist.

In most cases you'll want all these examples to be classes, but I've used simple functions here to keep the code samples brief.

Request rate limiter

This uses a basic token bucket algorithm and relies on the fact that Redis scripts execute atomically. No other operations can run between fetching the count and writing the new count.

skydan

#!/bin/bash

# This script will migrate schema and data from a SQLite3 database to PostgreSQL.
# Schema translation based on http://stackoverflow.com/a/4581921/1303625.
# Some column types are not handled (e.g blobs).
# 
# See also:
# - http://stackoverflow.com/questions/4581727/convert-sqlite-sql-dump-file-to-postgresql
# - https://gist.github.com/bittner/7368128

skydan

My largest Sidekiq application had a memory leak and I was able to find and fix it in just few hours spent on analyzing Ruby's heap. In this post I'll show my profiling setup.

As you might know Ruby 2.1 introduced a few great changes to ObjectSpace, so now it's much easier to find a line of code that is allocating too many objects. Here is great post explaining how it's working.

I was too lazy to set up some seeding and run it locally, so I checked that test suite passes when profiling is enabled and pushed debugging to production. Production environment also suited me better since my jobs data can't be fully random generated.

So, in order to profile your worker, add this to your Sidekiq configuration:

if ENV["PROFILE"]

skydan

#System Design Cheatsheet

Picking the right architecture = Picking the right battles + Managing trade-offs

##Basic Steps

1. Clarify and agree on the scope of the system

• User cases (description of sequences of events that, taken together, lead to a system doing something useful)
  • Who is going to use it?
  • How are they going to use it?

skydan

Setting up a SSL Cert from Comodo

I use Namecheap.com as a registrar, and they resale SSL Certs from a number of other companies, including Comodo.

These are the steps I went through to set up an SSL cert.

Purchase the cert

skydan

#!/usr/bin/env ruby

# Do sanity check on imported data by comparing the number of rows in each table

require 'pg'
require 'mysql2'

tables = %w( { list the talbes to compare here  })

p = PG.connect(dbname:'canvas', host: 'postgres.ocad.ca', user: 'canvas_user')

skydan

1. Edit /etc/sudoers

https://rvm.io/integration/sudo

2. Custom execute command

# deploy.rb
 desc "Export Upstart script"

skydan

cd ~
sudo apt-get update
sudo apt-get install unzip curl python-software-properties -y
#sudo add-apt-repository "deb http://archive.canonical.com/ lucid partner"
sudo add-apt-repository ppa:ferramroberto/java
sudo apt-get update
sudo apt-get install sun-java6-jre sun-java6-plugin -y

wget https://github.com/downloads/elasticsearch/elasticsearch/elasticsearch-0.18.7.tar.gz -O elasticsearch.tar.gz
tar -xf elasticsearch.tar.gz