pas256

Scaling your API with rate limiters

The following are examples of the four types rate limiters discussed in the accompanying blog post. In the examples below I've used pseudocode-like Ruby, so if you're unfamiliar with Ruby you should be able to easily translate this approach to other languages. Complete examples in Ruby are also provided later in this gist.

In most cases you'll want all these examples to be classes, but I've used simple functions here to keep the code samples brief.

Request rate limiter

This uses a basic token bucket algorithm and relies on the fact that Redis scripts execute atomically. No other operations can run between fetching the count and writing the new count.

pas256

#!/bin/bash

OAUTH2_TOKEN=<fill in your own OAUTH2 token>

OWNER=tiesmaster
REPO=Coolkit.Converters # retrieve this with: basename $(git config --get remote.origin.url) .git

curl https://api.github.com/repos/${OWNER}/${REPO}/branches/master \
    -H "Authorization: token $OAUTH2_TOKEN" \
    -H "Accept: application/vnd.github.loki-preview+json" \

pas256

import boto3
import argparse

from pprint import pprint as pp
from dateutil.tz import tzutc
from datetime import datetime, timedelta
from collections import defaultdict as dd

class IL(object):
    """

pas256

Keybase proof

I hereby claim:

• I am pas256 on github.
• I am pas256 (https://keybase.io/pas256) on keybase.
• I have a public key whose fingerprint is 7554 6406 36F3 4B85 38CE 049D 7900 D81F DCAF CA54

To claim this, I am signing this object:

pas256

[root@ip-10-0-12-67 ~]# mount /dev/xvdg /mnt
[root@ip-10-0-12-67 ~]# cd /mnt
[root@ip-10-0-12-67 mnt]# mount -t proc proc proc/
[root@ip-10-0-12-67 mnt]# mount -t sysfs sys sys/
[root@ip-10-0-12-67 mnt]# mount -o bind /dev dev/
[root@ip-10-0-12-67 mnt]# chroot /mnt
[root@ip-10-0-12-67 /]# cloud-init --debug init
2014-06-28 00:06:39,123 - util.py[DEBUG]: Reading from /proc/cmdline (quiet=False)
2014-06-28 00:06:39,124 - util.py[DEBUG]: Read 27 bytes from /proc/cmdline
2014-06-28 00:06:39,124 - util.py[DEBUG]: Reading from /proc/uptime (quiet=False)

pas256

{
    "Outputs": {
        "InstanceAccess": {
            "Description": "Command to use to SSH to instance",
            "Value": {
                "Fn::Join": [
                    "",
                    [
                        "ssh -i ",
                        {

pas256

Traceback (most recent call last):
  File "/Users/pas/ansible/lib/ansible/runner/__init__.py", line 65, in _executor_hook
    result_queue.put(multiprocessing_runner._executor(host))
  File "/Users/pas/ansible/lib/ansible/runner/__init__.py", line 281, in _executor
    self.callbacks.on_unreachable(host, msg)
  File "/Users/pas/ansible/lib/ansible/callbacks.py", line 312, in on_unreachable
    msg = "fatal: [%s] => %s" % (host, results)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xbf in position 205: ordinal not in range(128)

pas256

garethbowles1
1:55 spoon16, here's the answer from allenxwang:
1:55 You need to use JDBCConfigurationSource together with DynamicConfiguration, where it can poll the configuration source at some interval. Then add DynamicConfiguration to ConfigurationManager:
1:55 DynamicConfiguration dynamicConfig = ... // create config with JDBCConfigurationSource
1:55 (AggregatedConfiguration) config = (AggregatedConfiguration) ConfigurationManager.getConfigInstance();
1:55 config.addConfiguration(dynamicConfig, "Database");
1:55 Then the fast property will get callback once the value is changed in the database.

pas256

# aminate -B ami-86e15bef asgard
Traceback (most recent call last):
  File "/usr/bin/aminate", line 9, in <module>
    load_entry_point('aminator==1.0.0', 'console_scripts', 'aminate')()
  File "/usr/lib/python2.6/site-packages/aminator/cli.py", line 51, in run
    sys.exit(Aminator(debug=args.debug).aminate())
  File "/usr/lib/python2.6/site-packages/aminator/core.py", line 52, in __init__
    log_per_package(self.config, 'per_package')
  File "/usr/lib/python2.6/site-packages/aminator/config.py", line 161, in log_per_package
    dictConfig(per_package_config.toDict())

pas256

# Support for up to 10 Cassandra nodes
initial_tokens:
  1:
    0: 0
  2:
    0: 0
    1: 85070591730234615865843651857942052864
  3:
    0: 0
    1: 56713727820156410577229101238628035242