Skip to content

Instantly share code, notes, and snippets.

@domq
Created September 16, 2015 18:01
Show Gist options
  • Save domq/e23e08fab098d915f88f to your computer and use it in GitHub Desktop.
Save domq/e23e08fab098d915f88f to your computer and use it in GitHub Desktop.
Logs from random proxy cluster member - Leaves cluster upon loss of quorum, I think?
core@c06 ~ $ journalctl -u etcd2 --since="2015-09-10"
Sep 09 04:25:53 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/09 04:25:53 proxy: marked endpoint http://192.168.11.1:2379 available
Sep 09 04:25:54 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/09 04:25:54 proxy: marked endpoint http://192.168.11.8:2379 available
[No news for two days, which is great]
[At this point, node #7 goes off-line]
Sep 11 14:11:09 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 14:11:09 proxy: failed to direct request to http://192.168.11.7:2379: net/http: request canceled while waiting for connection
Sep 11 14:11:09 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 14:11:09 proxy: marked endpoint http://192.168.11.7:2379 unavailable
Sep 11 14:11:10 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 14:11:10 proxy: failed to direct request to http://192.168.11.7:2379: net/http: request canceled while waiting for connection
Sep 11 14:11:14 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 14:11:14 proxy: marked endpoint http://192.168.11.7:2379 available
Sep 11 14:11:15 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 14:11:15 proxy: failed to direct request to http://192.168.11.7:2379: net/http: request canceled while waiting for connection
Sep 11 14:11:15 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 14:11:15 proxy: marked endpoint http://192.168.11.7:2379 unavailable
Sep 11 14:11:15 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 14:11:15 proxy: failed to direct request to http://192.168.11.7:2379: net/http: request canceled while waiting for connection
Sep 11 14:11:15 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 14:11:15 proxy: failed to direct request to http://192.168.11.7:2379: net/http: request canceled while waiting for connection
Sep 11 14:11:15 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 14:11:15 proxy: failed to direct request to http://192.168.11.7:2379: net/http: request canceled while waiting for connection
Sep 11 14:11:20 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 14:11:20 proxy: marked endpoint http://192.168.11.7:2379 available
Sep 11 14:11:21 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 14:11:21 proxy: failed to direct request to http://192.168.11.7:2379: net/http: request canceled while waiting for connection
Sep 11 14:11:21 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 14:11:21 proxy: marked endpoint http://192.168.11.7:2379 unavailable
Sep 11 14:11:23 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 14:11:23 proxy: failed to direct request to http://192.168.11.7:2379: dial tcp 192.168.11.7:2379: no route to host
Sep 11 14:11:23 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 14:11:23 proxy: failed to direct request to http://192.168.11.7:2379: dial tcp 192.168.11.7:2379: no route to host
Sep 11 14:11:26 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 14:11:26 proxy: marked endpoint http://192.168.11.7:2379 available
Sep 11 14:11:26 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 14:11:26 proxy: failed to direct request to http://192.168.11.7:2379: dial tcp 192.168.11.7:2379: no route to host
Sep 11 14:11:26 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 14:11:26 proxy: marked endpoint http://192.168.11.7:2379 unavailable
Sep 11 14:11:31 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 14:11:31 proxy: marked endpoint http://192.168.11.7:2379 available
[... Snip ...]
[ At this point, node #7 comes back on-line and this looks like it wreaks havoc on the cluster members ]
Sep 11 15:32:33 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:32:33 proxy: marked endpoint http://192.168.11.7:2379 available
Sep 11 15:32:34 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:32:34 proxy: failed to direct request to http://192.168.11.7:2379: net/http: request canceled while waiting for connection
Sep 11 15:32:34 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:32:34 proxy: marked endpoint http://192.168.11.7:2379 unavailable
Sep 11 15:32:37 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:32:37 proxy: failed to direct request to http://192.168.11.7:2379: dial tcp 192.168.11.7:2379: i/o timeout
Sep 11 15:32:37 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:32:37 proxy: marked endpoint http://192.168.11.7:2379 unavailable
Sep 11 15:32:39 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:32:39 proxy: marked endpoint http://192.168.11.7:2379 available
Sep 11 15:32:42 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:32:42 proxy: marked endpoint http://192.168.11.7:2379 available
Sep 11 15:32:45 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:32:45 proxy: failed to direct request to http://192.168.11.7:2379: dial tcp 192.168.11.7:2379: i/o timeout
Sep 11 15:32:45 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:32:45 proxy: marked endpoint http://192.168.11.7:2379 unavailable
Sep 11 15:32:45 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:32:45 proxy: failed to direct request to http://192.168.11.7:2379: dial tcp 192.168.11.7:2379: i/o timeout
Sep 11 15:32:50 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:32:50 proxy: marked endpoint http://192.168.11.7:2379 available
Sep 11 15:33:15 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:15 proxy: failed to direct request to http://192.168.11.1:2379: read tcp 192.168.11.1:2379: use of closed network connection
Sep 11 15:33:15 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:15 proxy: marked endpoint http://192.168.11.1:2379 unavailable
Sep 11 15:33:15 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:15 proxy: failed to direct request to http://192.168.11.1:2379: read tcp 192.168.11.1:2379: use of closed network connection
Sep 11 15:33:15 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:15 proxy: failed to direct request to http://192.168.11.8:2379: dial tcp 192.168.11.8:2379: connection refused
Sep 11 15:33:15 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:15 proxy: marked endpoint http://192.168.11.8:2379 unavailable
Sep 11 15:33:15 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:15 proxy: failed to direct request to http://192.168.11.1:2379: read tcp 192.168.11.1:2379: use of closed network connection
Sep 11 15:33:15 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:15 proxy: failed to direct request to http://192.168.11.8:2379: dial tcp 192.168.11.8:2379: connection refused
Sep 11 15:33:15 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:15 proxy: failed to direct request to http://192.168.11.8:2379: dial tcp 192.168.11.8:2379: connection refused
Sep 11 15:33:15 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:15 proxy: failed to direct request to http://192.168.11.8:2379: dial tcp 192.168.11.8:2379: connection refused
Sep 11 15:33:17 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:17 proxy: failed to direct request to http://192.168.11.7:2379: read tcp 192.168.11.7:2379: use of closed network connection
Sep 11 15:33:17 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:17 proxy: marked endpoint http://192.168.11.7:2379 unavailable
Sep 11 15:33:17 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:17 proxy: failed to direct request to http://192.168.11.3:2379: dial tcp 192.168.11.3:2379: connection refused
Sep 11 15:33:17 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:17 proxy: marked endpoint http://192.168.11.3:2379 unavailable
Sep 11 15:33:17 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:17 proxy: unable to get response from 2 endpoint(s)
Sep 11 15:33:17 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:17 proxy: zero endpoints currently available
Sep 11 15:33:17 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:17 proxy: zero endpoints currently available
Sep 11 15:33:17 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:17 proxy: zero endpoints currently available
Sep 11 15:33:17 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:17 proxy: zero endpoints currently available
Sep 11 15:33:18 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:18 proxy: zero endpoints currently available
Sep 11 15:33:18 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:18 proxy: zero endpoints currently available
Sep 11 15:33:18 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:18 proxy: zero endpoints currently available
Sep 11 15:33:18 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:18 proxy: zero endpoints currently available
Sep 11 15:33:18 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:18 proxy: zero endpoints currently available
Sep 11 15:33:18 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:18 proxy: zero endpoints currently available
Sep 11 15:33:18 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:18 proxy: zero endpoints currently available
Sep 11 15:33:18 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:18 proxy: zero endpoints currently available
Sep 11 15:33:18 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:18 proxy: zero endpoints currently available
Sep 11 15:33:18 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:18 proxy: zero endpoints currently available
Sep 11 15:33:18 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:18 proxy: zero endpoints currently available
Sep 11 15:33:18 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:18 proxy: zero endpoints currently available
Sep 11 15:33:19 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:19 proxy: zero endpoints currently available
Sep 11 15:33:19 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:19 proxy: zero endpoints currently available
Sep 11 15:33:19 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:19 proxy: zero endpoints currently available
Sep 11 15:33:19 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:19 proxy: zero endpoints currently available
Sep 11 15:33:19 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:19 proxy: zero endpoints currently available
Sep 11 15:33:19 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:19 proxy: zero endpoints currently available
Sep 11 15:33:19 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:19 proxy: zero endpoints currently available
Sep 11 15:33:19 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:19 proxy: zero endpoints currently available
Sep 11 15:33:19 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:19 proxy: zero endpoints currently available
Sep 11 15:33:19 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:19 proxy: zero endpoints currently available
Sep 11 15:33:19 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:19 proxy: zero endpoints currently available
Sep 11 15:33:19 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:19 proxy: zero endpoints currently available
Sep 11 15:33:19 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:19 proxy: zero endpoints currently available
Sep 11 15:33:19 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:19 proxy: zero endpoints currently available
Sep 11 15:33:19 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:19 proxy: zero endpoints currently available
Sep 11 15:33:19 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:19 proxy: zero endpoints currently available
Sep 11 15:33:19 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:19 proxy: zero endpoints currently available
Sep 11 15:33:19 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:19 proxy: zero endpoints currently available
Sep 11 15:33:20 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:20 proxy: zero endpoints currently available
Sep 11 15:33:20 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:20 proxy: zero endpoints currently available
Sep 11 15:33:20 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:20 proxy: zero endpoints currently available
Sep 11 15:33:20 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:20 proxy: zero endpoints currently available
Sep 11 15:33:20 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:20 proxy: zero endpoints currently available
Sep 11 15:33:20 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:20 proxy: zero endpoints currently available
Sep 11 15:33:20 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:20 proxy: marked endpoint http://192.168.11.1:2379 available
Sep 11 15:33:20 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:20 proxy: marked endpoint http://192.168.11.8:2379 available
Sep 11 15:33:21 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:21 proxy: failed to direct request to http://192.168.11.1:2379: dial tcp 192.168.11.1:2379: connection refused
Sep 11 15:33:21 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:21 proxy: marked endpoint http://192.168.11.1:2379 unavailable
Sep 11 15:33:21 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:21 proxy: failed to direct request to http://192.168.11.8:2379: dial tcp 192.168.11.8:2379: connection refused
Sep 11 15:33:21 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:21 proxy: marked endpoint http://192.168.11.8:2379 unavailable
Sep 11 15:33:21 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:21 proxy: unable to get response from 2 endpoint(s)
Sep 11 15:33:21 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:21 proxy: zero endpoints currently available
Sep 11 15:33:21 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:21 proxy: zero endpoints currently available
Sep 11 15:33:21 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:21 proxy: zero endpoints currently available
Sep 11 15:33:21 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:21 proxy: zero endpoints currently available
Sep 11 15:33:21 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:21 proxy: zero endpoints currently available
Sep 11 15:33:21 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:21 proxy: zero endpoints currently available
Sep 11 15:33:21 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:21 proxy: zero endpoints currently available
Sep 11 15:33:21 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:21 proxy: zero endpoints currently available
Sep 11 15:33:21 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:21 proxy: zero endpoints currently available
Sep 11 15:33:21 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:21 proxy: zero endpoints currently available
Sep 11 15:33:21 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:21 proxy: zero endpoints currently available
Sep 11 15:33:22 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:22 proxy: zero endpoints currently available
Sep 11 15:33:22 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:22 proxy: zero endpoints currently available
Sep 11 15:33:22 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:22 proxy: marked endpoint http://192.168.11.7:2379 available
Sep 11 15:33:22 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:22 proxy: marked endpoint http://192.168.11.3:2379 available
Sep 11 15:33:26 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:26 proxy: marked endpoint http://192.168.11.1:2379 available
Sep 11 15:33:26 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:26 proxy: marked endpoint http://192.168.11.8:2379 available
Sep 11 15:33:30 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:30 proxy: failed to direct request to http://192.168.11.1:2379: read tcp 192.168.11.1:2379: use of closed network connection
Sep 11 15:33:30 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:30 proxy: marked endpoint http://192.168.11.1:2379 unavailable
Sep 11 15:33:30 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:30 proxy: failed to direct request to http://192.168.11.1:2379: read tcp 192.168.11.1:2379: use of closed network connection
Sep 11 15:33:30 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:30 proxy: failed to direct request to http://192.168.11.1:2379: read tcp 192.168.11.1:2379: use of closed network connection
Sep 11 15:33:30 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:30 proxy: failed to direct request to http://192.168.11.1:2379: read tcp 192.168.11.1:2379: use of closed network connection
Sep 11 15:33:30 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:30 proxy: failed to direct request to http://192.168.11.1:2379: read tcp 192.168.11.1:2379: use of closed network connection
Sep 11 15:33:31 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:31 proxy: failed to direct request to http://192.168.11.8:2379: read tcp 192.168.11.8:2379: use of closed network connection
Sep 11 15:33:31 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:31 proxy: marked endpoint http://192.168.11.8:2379 unavailable
Sep 11 15:33:31 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:31 proxy: failed to direct request to http://192.168.11.8:2379: read tcp 192.168.11.8:2379: use of closed network connection
Sep 11 15:33:31 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:31 proxy: failed to direct request to http://192.168.11.8:2379: read tcp 192.168.11.8:2379: use of closed network connection
Sep 11 15:33:31 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:31 proxy: failed to direct request to http://192.168.11.8:2379: read tcp 192.168.11.8:2379: use of closed network connection
Sep 11 15:33:31 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:31 proxy: failed to direct request to http://192.168.11.8:2379: read tcp 192.168.11.8:2379: use of closed network connection
Sep 11 15:33:32 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:32 proxy: failed to direct request to http://192.168.11.8:2379: read tcp 192.168.11.8:2379: use of closed network connection
Sep 11 15:33:32 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:32 proxy: failed to direct request to http://192.168.11.8:2379: read tcp 192.168.11.8:2379: use of closed network connection
Sep 11 15:33:32 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:32 proxy: failed to direct request to http://192.168.11.8:2379: EOF
Sep 11 15:33:32 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:32 proxy: failed to direct request to http://192.168.11.8:2379: EOF
Sep 11 15:33:32 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:32 proxy: failed to direct request to http://192.168.11.8:2379: EOF
Sep 11 15:33:32 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:32 proxy: failed to direct request to http://192.168.11.8:2379: EOF
Sep 11 15:33:32 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:32 proxy: failed to direct request to http://192.168.11.8:2379: EOF
Sep 11 15:33:33 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:33 proxy: failed to direct request to http://192.168.11.7:2379: read tcp 192.168.11.7:2379: use of closed network connection
Sep 11 15:33:33 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:33 proxy: marked endpoint http://192.168.11.7:2379 unavailable
Sep 11 15:33:33 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:33 proxy: failed to direct request to http://192.168.11.3:2379: dial tcp 192.168.11.3:2379: connection refused
Sep 11 15:33:33 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:33 proxy: marked endpoint http://192.168.11.3:2379 unavailable
Sep 11 15:33:33 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:33 proxy: unable to get response from 2 endpoint(s)
Sep 11 15:33:33 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:33 proxy: zero endpoints currently available
Sep 11 15:33:33 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:33 proxy: zero endpoints currently available
Sep 11 15:33:33 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:33 proxy: zero endpoints currently available
Sep 11 15:33:33 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:33 proxy: zero endpoints currently available
Sep 11 15:33:33 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:33 proxy: zero endpoints currently available
Sep 11 15:33:33 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:33 proxy: zero endpoints currently available
Sep 11 15:33:33 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:33 proxy: zero endpoints currently available
Sep 11 15:33:33 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:33 proxy: zero endpoints currently available
Sep 11 15:33:33 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:33 proxy: zero endpoints currently available
Sep 11 15:33:33 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:33 proxy: zero endpoints currently available
Sep 11 15:33:33 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:33 proxy: zero endpoints currently available
Sep 11 15:33:33 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:33 proxy: zero endpoints currently available
Sep 11 15:33:33 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:33 proxy: zero endpoints currently available
Sep 11 15:33:33 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:33 proxy: zero endpoints currently available
Sep 11 15:33:33 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:33 proxy: zero endpoints currently available
Sep 11 15:33:33 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:33 proxy: zero endpoints currently available
Sep 11 15:33:34 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:34 proxy: zero endpoints currently available
Sep 11 15:33:34 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:34 proxy: zero endpoints currently available
Sep 11 15:33:34 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:34 proxy: zero endpoints currently available
Sep 11 15:33:34 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:34 proxy: zero endpoints currently available
Sep 11 15:33:34 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:34 proxy: zero endpoints currently available
Sep 11 15:33:34 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:34 proxy: zero endpoints currently available
Sep 11 15:33:34 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:34 proxy: zero endpoints currently available
Sep 11 15:33:34 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:34 proxy: zero endpoints currently available
Sep 11 15:33:34 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:34 proxy: zero endpoints currently available
Sep 11 15:33:34 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:34 proxy: zero endpoints currently available
Sep 11 15:33:34 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:34 proxy: zero endpoints currently available
Sep 11 15:33:34 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:34 proxy: zero endpoints currently available
Sep 11 15:33:34 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:34 proxy: zero endpoints currently available
Sep 11 15:33:34 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:34 proxy: zero endpoints currently available
Sep 11 15:33:34 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:34 proxy: zero endpoints currently available
Sep 11 15:33:34 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:34 proxy: zero endpoints currently available
Sep 11 15:33:34 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:34 proxy: zero endpoints currently available
Sep 11 15:33:34 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:34 proxy: zero endpoints currently available
Sep 11 15:33:34 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:34 proxy: zero endpoints currently available
Sep 11 15:33:34 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:34 proxy: zero endpoints currently available
Sep 11 15:33:34 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:34 proxy: zero endpoints currently available
Sep 11 15:33:34 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:34 proxy: zero endpoints currently available
Sep 11 15:33:34 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:34 proxy: zero endpoints currently available
Sep 11 15:33:34 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:34 proxy: zero endpoints currently available
Sep 11 15:33:34 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:34 proxy: zero endpoints currently available
Sep 11 15:33:34 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:34 proxy: zero endpoints currently available
Sep 11 15:33:34 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:34 proxy: zero endpoints currently available
Sep 11 15:33:34 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:34 proxy: zero endpoints currently available
Sep 11 15:33:35 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:35 proxy: zero endpoints currently available
Sep 11 15:33:35 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:35 proxy: zero endpoints currently available
Sep 11 15:33:35 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:35 proxy: zero endpoints currently available
Sep 11 15:33:35 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:35 proxy: zero endpoints currently available
Sep 11 15:33:35 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:35 proxy: zero endpoints currently available
Sep 11 15:33:35 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:35 proxy: zero endpoints currently available
Sep 11 15:33:35 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:35 proxy: zero endpoints currently available
Sep 11 15:33:35 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:35 proxy: zero endpoints currently available
Sep 11 15:33:35 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:35 proxy: zero endpoints currently available
Sep 11 15:33:35 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:35 proxy: zero endpoints currently available
Sep 11 15:33:35 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:35 proxy: zero endpoints currently available
Sep 11 15:33:35 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:35 proxy: zero endpoints currently available
Sep 11 15:33:35 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:35 proxy: zero endpoints currently available
Sep 11 15:33:35 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:35 proxy: zero endpoints currently available
Sep 11 15:33:35 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:35 proxy: marked endpoint http://192.168.11.1:2379 available
Sep 11 15:33:35 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:35 proxy: failed to direct request to http://192.168.11.1:2379: dial tcp 192.168.11.1:2379: connection refused
Sep 11 15:33:35 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:35 proxy: marked endpoint http://192.168.11.1:2379 unavailable
Sep 11 15:33:35 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:35 proxy: unable to get response from 1 endpoint(s)
Sep 11 15:33:35 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:35 proxy: zero endpoints currently available
Sep 11 15:33:35 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:35 proxy: zero endpoints currently available
Sep 11 15:33:35 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:35 proxy: zero endpoints currently available
Sep 11 15:33:35 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:35 proxy: zero endpoints currently available
Sep 11 15:33:35 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:35 proxy: zero endpoints currently available
Sep 11 15:33:35 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:35 proxy: zero endpoints currently available
Sep 11 15:33:35 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:35 proxy: zero endpoints currently available
Sep 11 15:33:36 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:36 proxy: zero endpoints currently available
Sep 11 15:33:36 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:36 proxy: zero endpoints currently available
Sep 11 15:33:36 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:36 proxy: zero endpoints currently available
Sep 11 15:33:36 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:36 proxy: zero endpoints currently available
Sep 11 15:33:36 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:36 proxy: zero endpoints currently available
Sep 11 15:33:36 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:36 proxy: zero endpoints currently available
Sep 11 15:33:36 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:36 proxy: marked endpoint http://192.168.11.8:2379 available
Sep 11 15:33:37 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:37 proxy: failed to direct request to http://192.168.11.8:2379: dial tcp 192.168.11.8:2379: connection refused
Sep 11 15:33:37 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:37 proxy: marked endpoint http://192.168.11.8:2379 unavailable
Sep 11 15:33:37 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:37 proxy: unable to get response from 1 endpoint(s)
Sep 11 15:33:37 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:37 proxy: zero endpoints currently available
Sep 11 15:33:37 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:37 proxy: zero endpoints currently available
Sep 11 15:33:37 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:37 proxy: zero endpoints currently available
Sep 11 15:33:37 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:37 proxy: zero endpoints currently available
Sep 11 15:33:37 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:37 proxy: zero endpoints currently available
Sep 11 15:33:37 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:37 proxy: zero endpoints currently available
Sep 11 15:33:37 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:37 proxy: zero endpoints currently available
Sep 11 15:33:37 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:37 etcdserver: could not get cluster response from http://192.168.11.1:2380: Get http://192.168.11.1:2380/members: dial tcp 192.168.11.1:
Sep 11 15:33:37 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:37 etcdserver: could not get cluster response from http://192.168.11.3:2380: Get http://192.168.11.3:2380/members: dial tcp 192.168.11.3:
Sep 11 15:33:37 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:37 proxy: zero endpoints currently available
Sep 11 15:33:37 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:37 proxy: zero endpoints currently available
Sep 11 15:33:37 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:37 proxy: updated peer urls in cluster file from [http://192.168.11.1:2380 http://192.168.11.3:2380 http://192.168.11.7:2380 http://192.1
Sep 11 15:33:38 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:38 proxy: failed to direct request to http://192.168.11.7:2379: read tcp 192.168.11.7:2379: use of closed network connection
Sep 11 15:33:38 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:38 proxy: marked endpoint http://192.168.11.7:2379 unavailable
Sep 11 15:33:38 c06.ne.cloud.epfl.ch etcd2[3406]: 2015/09/11 15:33:38 proxy: unable to get response from 1 endpoint(s)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment