Skip to content

Instantly share code, notes, and snippets.

Consul Consistency

As Kyle brought up, Consul at the moment has a single known case of a potential inconsistency (Could be unknown cases lurking). Currently Consul works by electing a leader, who "leases" the position for LeaderLeaseTimeout interval. At each interval, it checks that a quorum of nodes still believes it to be the leader. At the same time, if a follower does not hear from the leader within randomInterva(HeartbeatTimeout, 2 * HeartbeatTimeout), it will start a new election.

armon:~/projects/consul-demo-tf/tf (master) $ TF_LOG=1 terraform plan
2014/10/15 19:51:31 Detected home directory from env var: /Users/armon
2014/10/15 19:51:31 [DEBUG] Discoverd plugin: aws = /Users/armon/projects/go/bin/terraform-provider-aws
2014/10/15 19:51:31 [DEBUG] Discoverd plugin: cloudflare = /Users/armon/projects/go/bin/terraform-provider-cloudflare
2014/10/15 19:51:31 [DEBUG] Discoverd plugin: consul = /Users/armon/projects/go/bin/terraform-provider-consul
2014/10/15 19:51:31 [DEBUG] Discoverd plugin: digitalocean = /Users/armon/projects/go/bin/terraform-provider-digitalocean
2014/10/15 19:51:31 [DEBUG] Discoverd plugin: dnsimple = /Users/armon/projects/go/bin/terraform-provider-dnsimple
2014/10/15 19:51:31 [DEBUG] Discoverd plugin: google = /Users/armon/projects/go/bin/terraform-provider-google
2014/10/15 19:51:31 [DEBUG] Discoverd plugin: heroku = /Users/armon/projects/go/bin/terraform-provider-heroku
2014/10/15 19:51:31 [DEBUG] Discoverd plugin: mailgun = /Users/armon/projects/go/bin/terraform-
@armon
armon / reap.sh
Created February 21, 2015 01:10
Reap Graphite nodes using Consul members
#!/bin/bash
# Store the live members
consul members | grep alive | awk '{ print $1 }' > /tmp/alive.txt
# Clean-up the collectd metrics
cd /data/graphite/whisper/collectd
ls | awk '{print substr($1, 0, index($1, "_node_")) }' > /tmp/monitored.txt
for NODE in `cat /tmp/monitored.txt`; do if grep -q $NODE /tmp/alive.txt; then echo $NODE alive; else echo $NODE dead; sudo rm -Rf ${NODE}_node_*; fi; done
@armon
armon / consul-tunes.md
Last active September 27, 2020 17:54
Running Consul as a global K/V store

Simplest way to do this with Consul is to run a single "global" datacenter.

This means the timing for the LAN gossip need to be tuned to be WAN appropriate. In consul/config.go (https://github.com/hashicorp/consul/blob/master/consul/config.go#L267) Do something like:

// Make the 'LAN' more forgiving for latency spikes
conf.SerfLANConfig.MemberlistConfig = memberlist.DefaultWANConfig()

Then we need to tune the Raft layer to be extremely forgiving.

@armon
armon / benchmark.go
Created November 1, 2013 00:26
scrypt parameter benchmarks
package main
import "code.google.com/p/go.crypto/scrypt"
import "fmt"
import "time"
func main() {
n_vals := []int{4 * 1024, 8 * 1024, 16 * 1024, 32 * 1024}
r_vals := []int{6, 7, 8, 9, 10}
p_vals := []int{1, 2, 3, 4}
@armon
armon / gist:a8f90ab7f50159ac3cc2
Created November 10, 2014 18:38
SmartStack vs Consul
Sent 5/1/2014
Hey Igor,
Glad you did a write up! I’m one of the authors of Consul. You mention we get some
things wrong about SmartStack, but we would love to get that corrected. The website
is generated from this file:
https://github.com/hashicorp/consul/blob/master/website/source/intro/vs/smartstack.html.markdown
func readPath(name string) {
p := GetPolicy(name)
DoSomething(p)
}
func writePath(name string) {
p := GetPolicy(name)
LockManager.Lock(name, func() {
DoSomethign(p)
package main
import (
"github.com/miekg/dns"
"log"
"net"
"time"
)
const (
@armon
armon / security.md
Last active December 26, 2015 13:29
#!/bin/bash
set -e
CONTAINER=`docker run -d ubuntu:precise echo test`
OUTPUT=`docker logs $CONTAINER`
echo \"$OUTPUT\" should be \"test\"
CONTAINER=`docker run -d ubuntu:precise echo test`
sleep 1
OUTPUT=`docker logs $CONTAINER`