... or Why Pipelining Is Not That Easy
Golang Concurrency Patterns for brave and smart.
By @kachayev
package main | |
import ( | |
"fmt" | |
) | |
func decorator(f func(s string)) func(s string) { | |
return func(s string) { | |
fmt.Println("Started") |
- What do Etcd, Consul, and Zookeeper do? | |
- Service Registration: | |
- Host, port number, and sometimes authentication credentials, protocols, versions | |
numbers, and/or environment details. | |
- Service Discovery: | |
- Ability for client application to query the central registry to learn of service location. | |
- Consistent and durable general-purpose K/V store across distributed system. | |
- Some solutions support this better than others. | |
- Based on Paxos or some derivative (i.e. Raft) algorithm to quickly converge to a consistent state. | |
- Centralized locking can be based on this K/V store. |
#! /usr/bin/env python | |
import redis | |
import random | |
import pylibmc | |
import sys | |
r = redis.Redis(host = 'localhost', port = 6389) | |
mc = pylibmc.Client(['localhost:11222']) |
... or Why Pipelining Is Not That Easy
Golang Concurrency Patterns for brave and smart.
By @kachayev
/* | |
This is a go program to download pictures from a tumblr blog page. | |
To Build: | |
go build -o tumblr-download tumblr-download.go | |
To Run: | |
# download the photos on the first page of tumblr blog |
package main | |
/* | |
Channel and select {} illustrated by t9md | |
=========================================================== | |
sample program to understand goroutine communication via channel. | |
based on go-tour's example | |
https://tour.golang.org/concurrency/5 | |
*/ |
function words($value, $words = 100, $end = '...') | |
{ | |
preg_match('/^\s*+(?:\S++\s*+){1,'.$words.'}/u', $value, $matches); | |
if ( ! isset($matches[0]) || strlen($value) === strlen($matches[0])) return $value; | |
return rtrim($matches[0]).$end; | |
} | |
/* | |
Usage: |
By default when Nginx starts receiving a response from a FastCGI backend (such as PHP-FPM) it will buffer the response in memory before delivering it to the client. Any response larger than the set buffer size is saved to a temporary file on disk. This process is also explained at the Nginx ngx_http_fastcgi_module page document page.
Since disk is slow and memory is fast the aim is to get as many FastCGI responses passing through memory only. On the flip side we don't want to set an excessively large buffer as they are created and sized on a per request basis (it's not shared).
The related Nginx options are:
fastcgi_buffering
appeared in Nginx 1.5.6 (1.6.0 stable) and can be used to turn buffering completely on/off. It's on by default.fastcgi_buffer_size
](http://nginx.org/en/docs/http/ngx_htLatency Comparison Numbers | |
-------------------------- | |
L1 cache reference 0.5 ns | |
Branch mispredict 5 ns | |
L2 cache reference 7 ns 14x L1 cache | |
Mutex lock/unlock 25 ns | |
Main memory reference 100 ns 20x L2 cache, 200x L1 cache | |
Compress 1K bytes with Zippy 3,000 ns | |
Send 1K bytes over 1 Gbps network 10,000 ns 0.01 ms | |
Read 4K randomly from SSD* 150,000 ns 0.15 ms |
<?php | |
/* | |
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS | |
* "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT | |
* LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR | |
* A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT | |
* OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, | |
* SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT | |
* LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, |