gnat

Better Primary Keys / PK / id

Why not just use sequential ID's (SERIAL or AUTO INCREMENT) ?

• Scaling writes require partitions / shards at some level.
  • Avoid SERIAL and AUTO INCREMENT from start.
    • Servers have to ask "what's the next available id in the sequence?" = huge global contention / slow down.
    • Turns into a queue. OK for 1 server. Bad for many servers.
      • Vitess and FoundationDB can work around this using sequence counters per server. Vitess one has an awful DX. Not sure about FoundationDB.
  • Alternative: Use time or random.
• UUID bonus: Split, merge tables, and load any rows any time you wish, since rows are globally unique.

kalaspuffar

Installing vitess

First of we need the registry and mysql server installed so we can keep track on which services are available, download new services and also run the actual database.

sudo apt install -y default-mysql-server default-mysql-client etcd curl

Before we start this POC we disable the default services as we will be setting up our own.

ptarjan

Scaling your API with rate limiters

The following are examples of the four types rate limiters discussed in the accompanying blog post. In the examples below I've used pseudocode-like Ruby, so if you're unfamiliar with Ruby you should be able to easily translate this approach to other languages. Complete examples in Ruby are also provided later in this gist.

In most cases you'll want all these examples to be classes, but I've used simple functions here to keep the code samples brief.

Request rate limiter

This uses a basic token bucket algorithm and relies on the fact that Redis scripts execute atomically. No other operations can run between fetching the count and writing the new count.

alexey-milovidov

:) SELECT toDateTime(intDiv(toUInt32(EventTime), 5) * 5) AS k, count(), uniq(UserID) FROM hits_layer WHERE CounterID = 29761725 AND EventDate = today() AND EventTime >= now() - 600 GROUP BY k ORDER BY k

SELECT 
    toDateTime(intDiv(toUInt32(EventTime), 5) * 5) AS k, 
    count(), 
    uniq(UserID)
FROM hits_layer
WHERE (CounterID = 29761725) AND (EventDate = today()) AND (EventTime >= (now() - 600))
GROUP BY k
ORDER BY k ASC

azimidev

0815.ru
0815.ru0clickemail.com
0815.ry
0815.su
0845.ru
0clickemail.com
0-mail.com
0wnd.net
0wnd.org
10mail.com

pnc

Using OTP's observer (appmon replacement) remotely

$ ssh remote-host "epmd -names"
epmd: up and running on port 4369 with data:
name some_node at port 58769

Note the running on port for epmd itself and the port of the node you're interested in debugging. Reconnect to the remote host with these ports forwarded:

$ ssh -L 4369:localhost:4369 -L 58769:localhost:58769 remote-host

juriansluiman

<?php                                                                        
                                                                             
include 'vendor/autoload.php';

$pheanstalk = new Pheanstalk_Pheanstalk('127.0.0.1');
$pheanstalk->useTube('default');

$n     = 1000000;
$start = microtime(true);

danmackinlay

#! /bin/sh
### BEGIN INIT INFO
# Provides:          supervisord
# Required-Start:    $remote_fs
# Required-Stop:     $remote_fs
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: Example initscript
# Description:       This file should be used to construct scripts to be
#                    placed in /etc/init.d.