Created
December 1, 2020 16:06
-
-
Save james64/882086b05e207f31a9899ccc5ee1b710 to your computer and use it in GitHub Desktop.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Node ID: '3048cdda-bdc9-e93e-54eb-809e8c03ae05' | |
| Node name: 'consul-server-1' | |
| Cluster Addr: 10.233.69.201 (LAN: 8301, WAN: 8302) | |
| ==> Starting Consul agent... | |
| Version: '1.9.0' | |
| Datacenter: 'dc1' (Segment: '<all>') | |
| Server: true (Bootstrap: false) | |
| Client Addr: [0.0.0.0] (HTTP: 8500, HTTPS: -1, gRPC: -1, DNS: 8600) | |
| Encrypt: Gossip: true, TLS-Outgoing: false, TLS-Incoming: false, Auto-Encrypt-TLS: false | |
| ==> Log data will now stream in as it occurs: | |
| 2020-12-01T14:44:06.692Z [WARN] agent: bootstrap_expect > 0: expecting 3 servers | |
| 2020-12-01T14:44:06.702Z [WARN] agent.auto_config: bootstrap_expect > 0: expecting 3 servers | |
| 2020-12-01T14:44:06.726Z [INFO] agent.server.raft: restored from snapshot: id=222-1458580-1606818035001 | |
| 2020-12-01T14:44:06.885Z [INFO] agent.server.raft: initial configuration: index=1462077 servers="[{Suffrage:Voter ID:c8bb9a79-dcf1-edd3-66b9-851265043986 Address:10.233.67.53:8300} {Suffrage:Nonvoter ID:48b9b896-6bb2-839f-209f-978dc8c93bfd Address:10.233.68.21:8300}]" | |
| 2020-12-01T14:44:06.885Z [INFO] agent.server.raft: entering follower state: follower="Node at 10.233.69.201:8300 [Follower]" leader= | |
| 2020-12-01T14:44:06.887Z [INFO] agent.server.serf.wan: serf: EventMemberJoin: consul-server-1.dc1 10.233.69.201 | |
| 2020-12-01T14:44:06.891Z [INFO] agent.router: Initializing LAN area manager | |
| 2020-12-01T14:44:06.892Z [INFO] agent.server: Adding LAN server: server="consul-server-1 (Addr: tcp/10.233.69.201:8300) (DC: dc1)" | |
| 2020-12-01T14:44:06.892Z [INFO] agent.server: Raft data found, disabling bootstrap mode | |
| 2020-12-01T14:44:06.891Z [INFO] agent.server.serf.lan: serf: EventMemberJoin: consul-server-1 10.233.69.201 | |
| 2020-12-01T14:44:06.892Z [INFO] agent.server: Handled event for server in area: event=member-join server=consul-server-1.dc1 area=wan | |
| 2020-12-01T14:44:06.899Z [INFO] agent: Starting server: address=[::]:8500 network=tcp protocol=http | |
| 2020-12-01T14:44:06.899Z [WARN] agent: DEPRECATED Backwards compatibility with pre-1.9 metrics enabled. These metrics will be removed in a future version of Consul. Set `telemetry { disable_compat_1.9 = true }` to disable them. | |
| 2020-12-01T14:44:06.898Z [INFO] agent: Started DNS server: address=0.0.0.0:8600 network=udp | |
| 2020-12-01T14:44:06.899Z [INFO] agent: Started DNS server: address=0.0.0.0:8600 network=tcp | |
| 2020-12-01T14:44:06.899Z [INFO] agent: Retry join is supported for the following discovery methods: cluster=LAN discovery_methods="aliyun aws azure digitalocean gce k8s linode mdns os packet scaleway softlayer tencentcloud triton vsphere" | |
| 2020-12-01T14:44:06.900Z [INFO] agent: Joining cluster...: cluster=LAN | |
| 2020-12-01T14:44:06.900Z [INFO] agent: (LAN) joining: lan_addresses=[consul-server-0.consul-server.consul.svc, consul-server-1.consul-server.consul.svc, consul-server-2.consul-server.consul.svc] | |
| 2020-12-01T14:44:06.900Z [INFO] agent: started state syncer | |
| ==> Consul agent running! | |
| 2020-12-01T14:44:06.912Z [INFO] agent.server.serf.lan: serf: EventMemberJoin: worker2.k8s 10.233.68.20 | |
| 2020-12-01T14:44:06.912Z [INFO] agent.server.serf.lan: serf: EventMemberJoin: consul-server-0 10.233.67.53 | |
| 2020-12-01T14:44:06.912Z [INFO] agent.server: Adding LAN server: server="consul-server-0 (Addr: tcp/10.233.67.53:8300) (DC: dc1)" | |
| 2020-12-01T14:44:06.913Z [INFO] agent.server.serf.lan: serf: EventMemberJoin: worker3.k8s 10.233.67.183 | |
| 2020-12-01T14:44:06.913Z [INFO] agent.server.serf.lan: serf: EventMemberJoin: worker1.k8s 10.233.69.200 | |
| 2020-12-01T14:44:06.913Z [WARN] agent.server.memberlist.lan: memberlist: Refuting a dead message (from: consul-server-1) | |
| 2020-12-01T14:44:06.914Z [INFO] agent.server.serf.lan: serf: EventMemberJoin: consul-server-2 10.233.68.21 | |
| 2020-12-01T14:44:06.914Z [INFO] agent.server: Adding LAN server: server="consul-server-2 (Addr: tcp/10.233.68.21:8300) (DC: dc1)" | |
| 2020-12-01T14:44:06.915Z [INFO] agent.server.serf.wan: serf: EventMemberJoin: consul-server-2.dc1 10.233.68.21 | |
| 2020-12-01T14:44:06.915Z [INFO] agent.server.serf.wan: serf: EventMemberJoin: consul-server-0.dc1 10.233.67.53 | |
| 2020-12-01T14:44:06.915Z [WARN] agent.server.memberlist.wan: memberlist: Refuting a dead message (from: consul-server-1.dc1) | |
| 2020-12-01T14:44:06.915Z [INFO] agent.server: Handled event for server in area: event=member-join server=consul-server-2.dc1 area=wan | |
| 2020-12-01T14:44:06.915Z [INFO] agent.server: Handled event for server in area: event=member-join server=consul-server-0.dc1 area=wan | |
| 2020-12-01T14:44:06.925Z [WARN] agent.server.raft: failed to get previous log: previous-index=1462084 last-index=1462077 error="log not found" | |
| 2020-12-01T14:44:06.928Z [WARN] agent.server.memberlist.lan: memberlist: Failed to resolve consul-server-1.consul-server.consul.svc: lookup consul-server-1.consul-server.consul.svc on 10.233.0.3:53: no such host | |
| 2020-12-01T14:44:06.933Z [INFO] agent: (LAN) joined: number_of_nodes=2 | |
| 2020-12-01T14:44:06.933Z [INFO] agent: Join cluster completed. Synced with initial agents: cluster=LAN num_agents=2 | |
| 2020-12-01T14:44:07.215Z [INFO] agent: Synced node info | |
| 2020-12-01T14:44:07.220Z [INFO] agent: Synced service: service=devportal-be-8080 | |
| 2020-12-01T14:44:15.067Z [INFO] agent.server.serf.wan: serf: EventMemberLeave: consul-server-0.dc1 10.233.67.53 | |
| 2020-12-01T14:44:15.067Z [INFO] agent.server: Handled event for server in area: event=member-leave server=consul-server-0.dc1 area=wan | |
| 2020-12-01T14:44:18.880Z [INFO] agent.server.serf.lan: serf: EventMemberLeave: consul-server-0 10.233.67.53 | |
| 2020-12-01T14:44:18.880Z [INFO] agent.server: Removing LAN server: server="consul-server-0 (Addr: tcp/10.233.67.53:8300) (DC: dc1)" | |
| 2020-12-01T14:44:20.892Z [INFO] agent.server: New leader elected: payload=consul-server-2 | |
| 2020-12-01T14:44:21.218Z [INFO] agent: Synced check: check=service:devportal-be-8080 | |
| 2020-12-01T14:44:32.514Z [INFO] agent.server.memberlist.lan: memberlist: Updating address for left or failed node consul-server-0 from 10.233.67.53:8301 to 10.233.67.184:8301 | |
| 2020-12-01T14:44:32.514Z [INFO] agent.server.serf.lan: serf: EventMemberJoin: consul-server-0 10.233.67.184 | |
| 2020-12-01T14:44:32.514Z [INFO] agent.server: Adding LAN server: server="consul-server-0 (Addr: tcp/10.233.67.184:8300) (DC: dc1)" | |
| 2020-12-01T14:44:32.516Z [INFO] agent.server.memberlist.wan: memberlist: Updating address for left or failed node consul-server-0.dc1 from 10.233.67.53:8302 to 10.233.67.184:8302 | |
| 2020-12-01T14:44:32.516Z [INFO] agent.server.serf.wan: serf: EventMemberJoin: consul-server-0.dc1 10.233.67.184 | |
| 2020-12-01T14:44:32.516Z [INFO] agent.server: Handled event for server in area: event=member-join server=consul-server-0.dc1 area=wan | |
| 2020-12-01T15:01:43.894Z [INFO] agent.server.memberlist.lan: memberlist: Suspect worker3.k8s has failed, no acks received | |
| 2020-12-01T15:01:47.892Z [INFO] agent.server.memberlist.lan: memberlist: Suspect consul-server-0 has failed, no acks received | |
| 2020-12-01T15:01:47.895Z [INFO] agent.server.memberlist.lan: memberlist: Marking worker3.k8s as failed, suspect timeout reached (2 peer confirmations) | |
| 2020-12-01T15:01:47.895Z [INFO] agent.server.serf.lan: serf: EventMemberFailed: worker3.k8s 10.233.67.183 | |
| 2020-12-01T15:01:48.881Z [INFO] agent.server.serf.lan: serf: EventMemberFailed: consul-server-0 10.233.67.184 | |
| 2020-12-01T15:01:48.881Z [INFO] agent.server: Removing LAN server: server="consul-server-0 (Addr: tcp/10.233.67.184:8300) (DC: dc1)" | |
| 2020-12-01T15:01:50.251Z [INFO] agent.server.serf.lan: serf: EventMemberJoin: consul-server-0 10.233.67.184 | |
| 2020-12-01T15:01:50.251Z [INFO] agent.server: Adding LAN server: server="consul-server-0 (Addr: tcp/10.233.67.184:8300) (DC: dc1)" | |
| 2020-12-01T15:01:52.296Z [INFO] agent.server.serf.lan: serf: EventMemberJoin: worker3.k8s 10.233.67.183 | |
| 2020-12-01T15:02:13.892Z [INFO] agent.server.memberlist.lan: memberlist: Suspect worker3.k8s has failed, no acks received | |
| 2020-12-01T15:02:16.888Z [INFO] agent.server.memberlist.wan: memberlist: Suspect consul-server-0.dc1 has failed, no acks received | |
| 2020-12-01T15:02:17.880Z [INFO] agent.server.serf.lan: serf: EventMemberFailed: consul-server-0 10.233.67.184 | |
| 2020-12-01T15:02:17.880Z [INFO] agent.server: Removing LAN server: server="consul-server-0 (Addr: tcp/10.233.67.184:8300) (DC: dc1)" | |
| 2020-12-01T15:02:17.892Z [INFO] agent.server.memberlist.lan: memberlist: Suspect worker3.k8s has failed, no acks received | |
| 2020-12-01T15:02:17.893Z [INFO] agent.server.memberlist.lan: memberlist: Marking worker3.k8s as failed, suspect timeout reached (2 peer confirmations) | |
| 2020-12-01T15:02:17.893Z [INFO] agent.server.serf.lan: serf: EventMemberFailed: worker3.k8s 10.233.67.183 | |
| 2020-12-01T15:02:21.950Z [INFO] agent.server: Removing LAN server: server="consul-server-0 (Addr: tcp/10.233.67.184:8300) (DC: dc1)" | |
| 2020-12-01T15:02:21.950Z [INFO] agent.server.serf.lan: serf: EventMemberLeave (forced): consul-server-0 10.233.67.184 | |
| 2020-12-01T15:02:22.433Z [INFO] agent.server.serf.lan: serf: EventMemberJoin: worker3.k8s 10.233.67.183 | |
| 2020-12-01T15:02:22.433Z [INFO] agent.server.serf.lan: serf: EventMemberJoin: consul-server-0 10.233.67.184 | |
| 2020-12-01T15:02:22.433Z [INFO] agent.server: Adding LAN server: server="consul-server-0 (Addr: tcp/10.233.67.184:8300) (DC: dc1)" | |
| 2020-12-01T15:02:29.892Z [INFO] agent.server.memberlist.lan: memberlist: Suspect worker3.k8s has failed, no acks received | |
| 2020-12-01T15:02:32.747Z [INFO] agent.server.serf.lan: serf: EventMemberFailed: worker3.k8s 10.233.67.183 | |
| 2020-12-01T15:02:32.747Z [INFO] agent.server.memberlist.lan: memberlist: Marking worker3.k8s as failed, suspect timeout reached (2 peer confirmations) | |
| 2020-12-01T15:02:34.892Z [INFO] agent.server.memberlist.lan: memberlist: Suspect consul-server-0 has failed, no acks received | |
| 2020-12-01T15:02:36.889Z [WARN] agent.server.memberlist.wan: memberlist: Was able to connect to consul-server-0.dc1 but other probes failed, network may be misconfigured | |
| 2020-12-01T15:02:37.280Z [INFO] agent.server.serf.lan: serf: EventMemberFailed: consul-server-0 10.233.67.184 | |
| 2020-12-01T15:02:37.280Z [INFO] agent.server: Removing LAN server: server="consul-server-0 (Addr: tcp/10.233.67.184:8300) (DC: dc1)" | |
| 2020-12-01T15:02:40.811Z [WARN] agent.server.memberlist.lan: memberlist: Refuting a suspect message (from: worker3.k8s) | |
| 2020-12-01T15:02:40.893Z [INFO] agent.server.serf.lan: serf: EventMemberLeave (forced): consul-server-0 10.233.67.184 | |
| 2020-12-01T15:02:40.893Z [INFO] agent.server: Removing LAN server: server="consul-server-0 (Addr: tcp/10.233.67.184:8300) (DC: dc1)" | |
| 2020-12-01T15:02:40.896Z [INFO] agent.server.serf.lan: serf: EventMemberFailed: worker2.k8s 10.233.68.20 | |
| 2020-12-01T15:02:41.096Z [INFO] agent.server.serf.lan: serf: EventMemberJoin: worker2.k8s 10.233.68.20 | |
| 2020-12-01T15:02:45.832Z [WARN] agent.server.raft: rejecting vote request since we have a leader: from=10.233.67.184:8300 leader=10.233.68.21:8300 | |
| 2020-12-01T15:02:46.889Z [INFO] agent.server.memberlist.wan: memberlist: Marking consul-server-0.dc1 as failed, suspect timeout reached (0 peer confirmations) | |
| 2020-12-01T15:02:46.889Z [INFO] agent.server.serf.wan: serf: EventMemberFailed: consul-server-0.dc1 10.233.67.184 | |
| 2020-12-01T15:02:46.889Z [INFO] agent.server: Handled event for server in area: event=member-failed server=consul-server-0.dc1 area=wan | |
| 2020-12-01T15:02:50.745Z [INFO] agent.server.serf.wan: serf: EventMemberJoin: consul-server-0.dc1 10.233.67.184 | |
| 2020-12-01T15:02:50.745Z [INFO] agent.server: Handled event for server in area: event=member-join server=consul-server-0.dc1 area=wan | |
| 2020-12-01T15:02:52.063Z [WARN] agent.server.raft: rejecting vote request since we have a leader: from=10.233.67.184:8300 leader=10.233.68.21:8300 | |
| 2020-12-01T15:02:52.547Z [INFO] agent.server.serf.lan: serf: EventMemberJoin: worker3.k8s 10.233.67.183 | |
| 2020-12-01T15:02:52.548Z [INFO] agent.server.serf.lan: serf: EventMemberJoin: consul-server-0 10.233.67.184 | |
| 2020-12-01T15:02:52.548Z [INFO] agent.server: Adding LAN server: server="consul-server-0 (Addr: tcp/10.233.67.184:8300) (DC: dc1)" | |
| 2020-12-01T15:02:58.738Z [ERROR] agent.server.raft: failed to flush response: error="write tcp 10.233.69.201:8300->10.233.68.21:37574: write: broken pipe" | |
| 2020-12-01T15:02:58.747Z [INFO] agent.server: New leader elected: payload=consul-server-2 | |
| 2020-12-01T15:03:01.392Z [WARN] agent.server.raft: rejecting vote request since we have a leader: from=10.233.67.184:8300 leader=10.233.68.21:8300 | |
| 2020-12-01T15:03:04.747Z [INFO] agent.server: New leader elected: payload=consul-server-2 | |
| 2020-12-01T15:08:01.892Z [INFO] agent.server.memberlist.lan: memberlist: Suspect worker3.k8s has failed, no acks received | |
| 2020-12-01T15:08:05.892Z [INFO] agent.server.memberlist.lan: memberlist: Suspect worker3.k8s has failed, no acks received | |
| 2020-12-01T15:08:07.347Z [INFO] agent.server.serf.lan: serf: EventMemberFailed: worker3.k8s 10.233.67.183 | |
| 2020-12-01T15:08:07.947Z [INFO] agent.server.serf.lan: serf: EventMemberFailed: consul-server-0 10.233.67.184 | |
| 2020-12-01T15:08:07.947Z [INFO] agent.server: Removing LAN server: server="consul-server-0 (Addr: tcp/10.233.67.184:8300) (DC: dc1)" | |
| 2020-12-01T15:08:09.892Z [INFO] agent.server.memberlist.lan: memberlist: Suspect consul-server-0 has failed, no acks received | |
| 2020-12-01T15:08:10.747Z [WARN] agent.server.memberlist.lan: memberlist: Refuting a suspect message (from: consul-server-2) | |
| 2020-12-01T15:08:10.747Z [INFO] agent.server.serf.lan: serf: EventMemberJoin: worker3.k8s 10.233.67.183 | |
| 2020-12-01T15:08:10.747Z [INFO] agent.server.serf.lan: serf: EventMemberJoin: consul-server-0 10.233.67.184 | |
| 2020-12-01T15:08:10.747Z [INFO] agent.server: Adding LAN server: server="consul-server-0 (Addr: tcp/10.233.67.184:8300) (DC: dc1)" |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment