etcd-testing.md

This is just an example that will only work for specific network interfaces and addresses. Modify as needed.

Create data directories for etcd members:

mkdir -p e{0,1,2}

Remove data directories:

rm -rf e{0,1,2}/*

Create individual IP addresses for local etcd cluster testing (use different network interface name according to your specific configuration):

ip address add 192.168.56.30 dev enp0s3
ip address add 192.168.56.31 dev enp0s3
ip address add 192.168.56.32 dev enp0s3
ip address add 192.168.56.40 dev enp0s3
ip address add 192.168.56.41 dev enp0s3
ip address add 192.168.56.42 dev enp0s3

List addresses:

ip -4 address

Start 3-member etcd cluster on 192.168.56.3* addresses:

export A=3

for I in 0 1 2 ; do
  docker run \
    --rm \
    -d \
    --net=host \
    --name etcd${I} \
    --volume=$(pwd)/e${I}:/var/etcd/data \
    k8s.gcr.io/etcd:3.1.12 /usr/local/bin/etcd \
    --quota-backend-bytes=4294967296 \
    --data-dir=/var/etcd/data \
    --name=etcd${I} \
    --listen-client-urls=http://192.168.56.${A}${I}:2379 \
    --advertise-client-urls=http://192.168.56.${A}${I}:2379 \
    --listen-peer-urls=http://192.168.56.${A}${I}:2380 \
    --initial-cluster-token=etcd-cluster-k8s-cluster-1537322796 \
    --initial-advertise-peer-urls=http://192.168.56.${A}${I}:2380 \
    --initial-cluster-state=new \
    --initial-cluster=etcd0=http://192.168.56.${A}0:2380,etcd1=http://192.168.56.${A}1:2380,etcd2=http://192.168.56.${A}2:2380
done

Start CLI with client configured:

docker run --rm -it --net=host --name etcdclient \
-e ETCDCTL_PEERS=http://192.168.56.${A}0:2379,http://192.168.56.${A}1:2379,http://192.168.56.${A}2:2379 \
k8s.gcr.io/etcd:3.1.12 /bin/sh

Stop all the members:

docker stop etcd0 etcd1 etcd2

Now you can test different etcd availability and recovery scenarios by killing and restarting corresponding containers (do not use --rm flag if you want docker to keep containers when they are stopped).

If you want to see how cluster behaves if it is restarted on different IPs, run the same start command as above but replace A env var value with 4:

export A=4

for I in 0 1 2 ; do
  docker run \
    --rm \
    -d \
    --net=host \
    --name etcd${I} \
    --volume=$(pwd)/e${I}:/var/etcd/data \
    k8s.gcr.io/etcd:3.1.12 /usr/local/bin/etcd \
    --quota-backend-bytes=4294967296 \
    --data-dir=/var/etcd/data \
    --name=etcd${I} \
    --listen-client-urls=http://192.168.56.${A}${I}:2379 \
    --advertise-client-urls=http://192.168.56.${A}${I}:2379 \
    --listen-peer-urls=http://192.168.56.${A}${I}:2380 \
    --initial-cluster-token=etcd-cluster-k8s-cluster-1537322796 \
    --initial-advertise-peer-urls=http://192.168.56.${A}${I}:2380 \
    --initial-cluster-state=new \
    --initial-cluster=etcd0=http://192.168.56.${A}0:2380,etcd1=http://192.168.56.${A}1:2380,etcd2=http://192.168.56.${A}2:2380