Skip to content

Instantly share code, notes, and snippets.

View ntulip's full-sized avatar

Nick Tulip ntulip

View GitHub Profile
@ntulip
ntulip / cloudSettings
Last active July 1, 2019 12:34
Visual Studio Code Settings Sync Gist
{"lastUpload":"2019-07-01T12:34:16.329Z","extensionVersion":"v3.2.9"}

Scaling your API with rate limiters

The following are examples of the four types rate limiters discussed in the accompanying blog post. In the examples below I've used pseudocode-like Ruby, so if you're unfamiliar with Ruby you should be able to easily translate this approach to other languages. Complete examples in Ruby are also provided later in this gist.

In most cases you'll want all these examples to be classes, but I've used simple functions here to keep the code samples brief.

Request rate limiter

This uses a basic token bucket algorithm and relies on the fact that Redis scripts execute atomically. No other operations can run between fetching the count and writing the new count.

@ntulip
ntulip / message_queue_pipeline.py
Created June 29, 2016 19:10 — forked from azizmb/message_queue_pipeline.py
Scrapy pipeline to enque scraped items to message queue using carrot
from scrapy.xlib.pydispatch import dispatcher
from scrapy import signals
from scrapy.exceptions import DropItem
from scrapy.utils.serialize import ScrapyJSONEncoder
from carrot.connection import BrokerConnection
from carrot.messaging import Publisher
from twisted.internet.threads import deferToThread
@ntulip
ntulip / certchain.sh
Created January 18, 2016 18:18 — forked from joshenders/certchain.sh
Simple Bash function to visualize PKI chain-of-trust
function certchain() {
# Usage: certchain
# Display PKI chain-of-trust for a given domain
# GistID: https://gist.github.com/joshenders/cda916797665de69ebcd
if [[ "$#" -ne 1 ]]; then
echo "Usage: ${FUNCNAME} <ip|domain[:port]>"
return 1
fi
local host_port="$1"
{
"title": "Apache logs",
"services": {
"query": {
"list": {
"0": {
"query": "*",
"alias": "",
"color": "#7EB26D",
"id": 0,
@ntulip
ntulip / logs.sh
Created March 21, 2013 18:31
check your ubuntu logs
# http://ubuntuforums.org/showthread.php?t=2035442&s=2fd15b2a74f9f2ca29acab1ce5933614&p=12140524#post12140524
sudo grep 'warning\|oops\|segfault\|error' -iHnr /var/log/dmesg /var/log/syslog /var/log/kern.log /var/log/Xorg.0.log $HOME/.xsession-errors > $HOME/mylog.txt && nano $HOME/mylog.txt
@ntulip
ntulip / UUID.js
Created August 16, 2012 11:40
UUIDinJS
// UUID/Guid Generator - http://frugalcoder.us/post/2012/01/13/javascript-guid-uuid-generator.aspx
// use: UUID.create() or UUID.createSequential()
// convenience: UUID.empty, UUID.tryParse(string)
(function(w){
// From http://baagoe.com/en/RandomMusings/javascript/
// Johannes Baagøe <baagoe@baagoe.com>, 2010
function Mash() {
var n = 0xefc8249d;
var mash = function(data) {
@ntulip
ntulip / Ubuntu12-04
Created June 4, 2012 15:36 — forked from devilankur18/Ubuntu12-04
Graylog Setup in ubuntu 12.04
Grab the deb packages for Elasticsearch from http://www.elasticsearch.org/download/ and both the Graylog2 server and web interface from http://graylog2.org/download.
#install the elastic search from debian
dpkg -i elasticsearch-0.19.3.deb
References
http://imcol.in/category/monitoring/
@ntulip
ntulip / static.rb
Created June 4, 2012 15:33
Static Sites on Heroku
# From http://anti-pattern.com/2012/6/2/static-sites-on-heroku
# Gemfile
source "http://rubygems.org"
gem "rack"
# config.ru
require "rubygems"
require "bundler"
@ntulip
ntulip / Graphite Install Ubuntu 11.04 Natty
Created May 2, 2012 01:31
Graphite Install Ubuntu 11.04 Natty
####################################
# BASIC REQUIREMENTS
# http://graphite.wikidot.com/installation
# http://geek.michaelgrace.org/2011/09/how-to-install-graphite-on-ubuntu/
####################################
sudo apt-get update
wget http://launchpad.net/graphite/1.0/0.9.8/+download/graphite-web-0.9.8.tar.gz
wget http://launchpad.net/graphite/1.0/0.9.8/+download/carbon-0.9.8.tar.gz
wget http://launchpad.net/graphite/1.0/0.9.8/+download/whisper-0.9.8.tar.gz