gistfile1.txt

Sep 14 17:41:03 node1.mydomain.com corosync[558167]:   [KNET  ] rx: host: 3 link: 0 is up
Sep 14 17:41:03 node1.mydomain.com corosync[558167]:   [KNET  ] host: host: 3 (passive) best link: 0 (pri: 1)
Sep 14 17:41:03 node1.mydomain.com corosync[558167]:   [KNET  ] pmtud: PMTUD link change for host: 3 link: 0 from 469 to 1397
Sep 14 17:41:04 node1.mydomain.com corosync[558167]:   [QUORUM] Sync members[3]: 1 2 3
Sep 14 17:41:04 node1.mydomain.com corosync[558167]:   [QUORUM] Sync joined[1]: 3
Sep 14 17:41:04 node1.mydomain.com corosync[558167]:   [TOTEM ] A new membership (1.1d7) was formed. Members joined: 3
Sep 14 17:41:04 node1.mydomain.com pmxcfs[539995]: [dcdb] notice: members: 1/539995, 2/422073, 3/1325
Sep 14 17:41:04 node1.mydomain.com pmxcfs[539995]: [dcdb] notice: starting data syncronisation
Sep 14 17:41:04 node1.mydomain.com corosync[558167]:   [QUORUM] Members[3]: 1 2 3
Sep 14 17:41:04 node1.mydomain.com corosync[558167]:   [MAIN  ] Completed service synchronization, ready to provide service.
Sep 14 17:41:04 node1.mydomain.com pmxcfs[539995]: [dcdb] notice: cpg_send_message retried 1 times
Sep 14 17:41:04 node1.mydomain.com pmxcfs[539995]: [status] notice: members: 1/539995, 2/422073, 3/1325
Sep 14 17:41:04 node1.mydomain.com pmxcfs[539995]: [status] notice: starting data syncronisation
Sep 14 17:41:04 node1.mydomain.com pmxcfs[539995]: [dcdb] notice: received sync request (epoch 1/539995/0000001C)
Sep 14 17:41:04 node1.mydomain.com pmxcfs[539995]: [status] notice: received sync request (epoch 1/539995/0000001B)
Sep 14 17:41:04 node1.mydomain.com pmxcfs[539995]: [dcdb] notice: received all states
Sep 14 17:41:04 node1.mydomain.com pmxcfs[539995]: [dcdb] notice: leader is 1/539995
Sep 14 17:41:04 node1.mydomain.com pmxcfs[539995]: [dcdb] notice: synced members: 1/539995, 2/422073
Sep 14 17:41:04 node1.mydomain.com pmxcfs[539995]: [dcdb] notice: start sending inode updates
Sep 14 17:41:04 node1.mydomain.com pmxcfs[539995]: [dcdb] notice: sent all (7) updates
Sep 14 17:41:04 node1.mydomain.com pmxcfs[539995]: [dcdb] notice: all data is up to date
Sep 14 17:41:04 node1.mydomain.com pmxcfs[539995]: [status] notice: received all states
Sep 14 17:41:04 node1.mydomain.com pmxcfs[539995]: [status] notice: all data is up to date
Sep 14 17:41:04 node1.mydomain.com corosync[558167]:   [TOTEM ] Retransmit List: 11 12 13
Sep 14 17:41:04 node1.mydomain.com corosync[558167]:   [TOTEM ] Retransmit List: 11 12 13
Sep 14 17:41:04 node1.mydomain.com corosync[558167]:   [TOTEM ] Retransmit List: 11 12 13
Sep 14 17:41:05 node1.mydomain.com corosync[558167]:   [TOTEM ] Retransmit List: 11 12 13
Sep 14 17:41:05 node1.mydomain.com corosync[558167]:   [TOTEM ] Retransmit List: 11 12 13
Sep 14 17:41:05 node1.mydomain.com corosync[558167]:   [TOTEM ] Retransmit List: 11 12 13
Sep 14 17:41:05 node1.mydomain.com corosync[558167]:   [TOTEM ] Retransmit List: 11 12 13
Sep 14 17:41:06 node1.mydomain.com corosync[558167]:   [TOTEM ] Retransmit List: 11 12 13
Sep 14 17:41:06 node1.mydomain.com corosync[558167]:   [TOTEM ] Retransmit List: 11 12 13
Sep 14 17:41:07 node1.mydomain.com corosync[558167]:   [TOTEM ] Retransmit List: 11 12 13
Sep 14 17:41:07 node1.mydomain.com corosync[558167]:   [TOTEM ] Retransmit List: 11 12 13
Sep 14 17:41:08 node1.mydomain.com corosync[558167]:   [TOTEM ] Retransmit List: 11 12 13
Sep 14 17:41:09 node1.mydomain.com corosync[558167]:   [TOTEM ] Retransmit List: 11 12 13
Sep 14 17:41:09 node1.mydomain.com corosync[558167]:   [TOTEM ] Retransmit List: 11 12 13
Sep 14 17:41:09 node1.mydomain.com corosync[558167]:   [TOTEM ] Retransmit List: 11 12 13
Sep 14 17:41:10 node1.mydomain.com corosync[558167]:   [TOTEM ] Retransmit List: 11 12 13
Sep 14 17:41:10 node1.mydomain.com corosync[558167]:   [TOTEM ] Retransmit List: 11 12 13 22
Sep 14 17:41:10 node1.mydomain.com corosync[558167]:   [TOTEM ] Retransmit List: 11 12 13 22
Sep 14 17:41:10 node1.mydomain.com corosync[558167]:   [TOTEM ] Retransmit List: 11 12 13 22
Sep 14 17:41:10 node1.mydomain.com corosync[558167]:   [TOTEM ] Retransmit List: 11 12 13 22
Sep 14 17:41:10 node1.mydomain.com corosync[558167]:   [TOTEM ] Retransmit List: 11 12 13 22
Sep 14 17:41:10 node1.mydomain.com corosync[558167]:   [TOTEM ] Retransmit List: 11 12 13 22
Sep 14 17:41:10 node1.mydomain.com corosync[558167]:   [TOTEM ] Retransmit List: 11 12 13 22
Sep 14 17:41:10 node1.mydomain.com corosync[558167]:   [TOTEM ] Retransmit List: 11 12 13 22
Sep 14 17:41:10 node1.mydomain.com corosync[558167]:   [TOTEM ] Retransmit List: 11 12 13 22







Sep 14 17:41:01 node2.mydomain.com CRON[438174]: pam_unix(cron:session): session opened for user root(uid=0) by (uid=0)
Sep 14 17:41:01 node2.mydomain.com CRON[438175]: (root) CMD (/root/server-configs/cron/minute.sh > /dev/null 2>&1)
Sep 14 17:41:01 node2.mydomain.com CRON[438174]: pam_unix(cron:session): session closed for user root
Sep 14 17:41:03 node2.mydomain.com corosync[436735]:   [KNET  ] rx: host: 3 link: 0 is up
Sep 14 17:41:03 node2.mydomain.com corosync[436735]:   [KNET  ] host: host: 3 (passive) best link: 0 (pri: 1)
Sep 14 17:41:03 node2.mydomain.com corosync[436735]:   [KNET  ] pmtud: PMTUD link change for host: 3 link: 0 from 469 to 1397
Sep 14 17:41:04 node2.mydomain.com corosync[436735]:   [QUORUM] Sync members[3]: 1 2 3
Sep 14 17:41:04 node2.mydomain.com corosync[436735]:   [QUORUM] Sync joined[1]: 3
Sep 14 17:41:04 node2.mydomain.com corosync[436735]:   [TOTEM ] A new membership (1.1d7) was formed. Members joined: 3
Sep 14 17:41:04 node2.mydomain.com pmxcfs[422073]: [dcdb] notice: members: 1/539995, 2/422073, 3/1325
Sep 14 17:41:04 node2.mydomain.com pmxcfs[422073]: [dcdb] notice: starting data syncronisation
Sep 14 17:41:04 node2.mydomain.com pmxcfs[422073]: [status] notice: members: 1/539995, 2/422073, 3/1325
Sep 14 17:41:04 node2.mydomain.com pmxcfs[422073]: [status] notice: starting data syncronisation
Sep 14 17:41:04 node2.mydomain.com corosync[436735]:   [QUORUM] Members[3]: 1 2 3
Sep 14 17:41:04 node2.mydomain.com corosync[436735]:   [MAIN  ] Completed service synchronization, ready to provide service.
Sep 14 17:41:04 node2.mydomain.com pmxcfs[422073]: [dcdb] notice: received sync request (epoch 1/539995/0000001C)
Sep 14 17:41:04 node2.mydomain.com pmxcfs[422073]: [status] notice: received sync request (epoch 1/539995/0000001B)







Sep 14 17:41:00 node3.mydomain.com systemd[1]: Starting Corosync Cluster Engine...
Sep 14 17:41:00 node3.mydomain.com corosync[8654]:   [MAIN  ] Corosync Cluster Engine 3.1.5 starting up
Sep 14 17:41:00 node3.mydomain.com corosync[8654]:   [MAIN  ] Corosync built-in features: dbus monitoring watchdog systemd xmlconf vqsim nozzle snmp pie relro bindnow
Sep 14 17:41:00 node3.mydomain.com corosync[8654]:   [TOTEM ] Initializing transport (Kronosnet).
Sep 14 17:41:01 node3.mydomain.com corosync[8654]:   [TOTEM ] totemknet initialized
Sep 14 17:41:01 node3.mydomain.com corosync[8654]:   [KNET  ] common: crypto_nss.so has been loaded from /usr/lib/x86_64-linux-gnu/kronosnet/crypto_nss.so
Sep 14 17:41:01 node3.mydomain.com corosync[8654]:   [SERV  ] Service engine loaded: corosync configuration map access [0]
Sep 14 17:41:01 node3.mydomain.com corosync[8654]:   [QB    ] server name: cmap
Sep 14 17:41:01 node3.mydomain.com corosync[8654]:   [SERV  ] Service engine loaded: corosync configuration service [1]
Sep 14 17:41:01 node3.mydomain.com corosync[8654]:   [QB    ] server name: cfg
Sep 14 17:41:01 node3.mydomain.com corosync[8654]:   [SERV  ] Service engine loaded: corosync cluster closed process group service v1.01 [2]
Sep 14 17:41:01 node3.mydomain.com corosync[8654]:   [QB    ] server name: cpg
Sep 14 17:41:01 node3.mydomain.com corosync[8654]:   [SERV  ] Service engine loaded: corosync profile loading service [4]
Sep 14 17:41:01 node3.mydomain.com corosync[8654]:   [SERV  ] Service engine loaded: corosync resource monitoring service [6]
Sep 14 17:41:01 node3.mydomain.com corosync[8654]:   [WD    ] Watchdog not enabled by configuration
Sep 14 17:41:01 node3.mydomain.com corosync[8654]:   [WD    ] resource load_15min missing a recovery key.
Sep 14 17:41:01 node3.mydomain.com corosync[8654]:   [WD    ] resource memory_used missing a recovery key.
Sep 14 17:41:01 node3.mydomain.com corosync[8654]:   [WD    ] no resources configured.
Sep 14 17:41:01 node3.mydomain.com corosync[8654]:   [SERV  ] Service engine loaded: corosync watchdog service [7]
Sep 14 17:41:01 node3.mydomain.com corosync[8654]:   [QUORUM] Using quorum provider corosync_votequorum
Sep 14 17:41:01 node3.mydomain.com corosync[8654]:   [SERV  ] Service engine loaded: corosync vote quorum service v1.0 [5]
Sep 14 17:41:01 node3.mydomain.com corosync[8654]:   [QB    ] server name: votequorum
Sep 14 17:41:01 node3.mydomain.com corosync[8654]:   [SERV  ] Service engine loaded: corosync cluster quorum service v0.1 [3]
Sep 14 17:41:01 node3.mydomain.com corosync[8654]:   [QB    ] server name: quorum
Sep 14 17:41:01 node3.mydomain.com corosync[8654]:   [TOTEM ] Configuring link 0
Sep 14 17:41:01 node3.mydomain.com corosync[8654]:   [TOTEM ] Configured link number 0: local addr: 54.38.178.59, port=5405
Sep 14 17:41:01 node3.mydomain.com corosync[8654]:   [KNET  ] host: host: 1 (passive) best link: 0 (pri: 0)
Sep 14 17:41:01 node3.mydomain.com corosync[8654]:   [KNET  ] host: host: 1 has no active links
Sep 14 17:41:01 node3.mydomain.com corosync[8654]:   [KNET  ] host: host: 1 (passive) best link: 0 (pri: 1)
Sep 14 17:41:01 node3.mydomain.com corosync[8654]:   [KNET  ] host: host: 1 has no active links
Sep 14 17:41:01 node3.mydomain.com corosync[8654]:   [KNET  ] host: host: 1 (passive) best link: 0 (pri: 1)
Sep 14 17:41:01 node3.mydomain.com corosync[8654]:   [KNET  ] host: host: 1 has no active links
Sep 14 17:41:01 node3.mydomain.com corosync[8654]:   [KNET  ] host: host: 2 (passive) best link: 0 (pri: 0)
Sep 14 17:41:01 node3.mydomain.com corosync[8654]:   [KNET  ] host: host: 2 has no active links
Sep 14 17:41:01 node3.mydomain.com corosync[8654]:   [KNET  ] host: host: 2 (passive) best link: 0 (pri: 1)
Sep 14 17:41:01 node3.mydomain.com corosync[8654]:   [KNET  ] host: host: 2 has no active links
Sep 14 17:41:01 node3.mydomain.com corosync[8654]:   [KNET  ] host: host: 2 (passive) best link: 0 (pri: 1)
Sep 14 17:41:01 node3.mydomain.com corosync[8654]:   [KNET  ] host: host: 2 has no active links
Sep 14 17:41:01 node3.mydomain.com corosync[8654]:   [KNET  ] host: host: 3 (passive) best link: 0 (pri: 0)
Sep 14 17:41:01 node3.mydomain.com corosync[8654]:   [KNET  ] host: host: 3 has no active links
Sep 14 17:41:01 node3.mydomain.com corosync[8654]:   [QUORUM] Sync members[1]: 3
Sep 14 17:41:01 node3.mydomain.com corosync[8654]:   [QUORUM] Sync joined[1]: 3
Sep 14 17:41:01 node3.mydomain.com corosync[8654]:   [TOTEM ] A new membership (3.1c2) was formed. Members joined: 3
Sep 14 17:41:01 node3.mydomain.com corosync[8654]:   [QUORUM] Members[1]: 3
Sep 14 17:41:01 node3.mydomain.com corosync[8654]:   [MAIN  ] Completed service synchronization, ready to provide service.
Sep 14 17:41:01 node3.mydomain.com systemd[1]: Started Corosync Cluster Engine.
Sep 14 17:41:02 node3.mydomain.com pmxcfs[1325]: [status] notice: update cluster info (cluster name  opendle, version = 14)
Sep 14 17:41:02 node3.mydomain.com pmxcfs[1325]: [dcdb] notice: members: 3/1325
Sep 14 17:41:02 node3.mydomain.com pmxcfs[1325]: [dcdb] notice: all data is up to date
Sep 14 17:41:02 node3.mydomain.com pmxcfs[1325]: [status] notice: members: 3/1325
Sep 14 17:41:02 node3.mydomain.com pmxcfs[1325]: [status] notice: all data is up to date
Sep 14 17:41:03 node3.mydomain.com corosync[8654]:   [KNET  ] rx: host: 1 link: 0 is up
Sep 14 17:41:03 node3.mydomain.com corosync[8654]:   [KNET  ] host: host: 1 (passive) best link: 0 (pri: 1)
Sep 14 17:41:03 node3.mydomain.com corosync[8654]:   [KNET  ] rx: host: 2 link: 0 is up
Sep 14 17:41:03 node3.mydomain.com corosync[8654]:   [KNET  ] host: host: 2 (passive) best link: 0 (pri: 1)
Sep 14 17:41:03 node3.mydomain.com corosync[8654]:   [KNET  ] pmtud: PMTUD link change for host: 2 link: 0 from 469 to 1397
Sep 14 17:41:03 node3.mydomain.com corosync[8654]:   [KNET  ] pmtud: PMTUD link change for host: 1 link: 0 from 469 to 1397
Sep 14 17:41:03 node3.mydomain.com corosync[8654]:   [KNET  ] pmtud: Global data MTU changed to: 1397
Sep 14 17:41:04 node3.mydomain.com corosync[8654]:   [QUORUM] Sync members[3]: 1 2 3
Sep 14 17:41:04 node3.mydomain.com corosync[8654]:   [QUORUM] Sync joined[2]: 1 2
Sep 14 17:41:04 node3.mydomain.com corosync[8654]:   [TOTEM ] A new membership (1.1d7) was formed. Members joined: 1 2
Sep 14 17:41:04 node3.mydomain.com pmxcfs[1325]: [dcdb] notice: members: 1/539995, 2/422073, 3/1325
Sep 14 17:41:04 node3.mydomain.com pmxcfs[1325]: [dcdb] notice: starting data syncronisation
Sep 14 17:41:04 node3.mydomain.com pmxcfs[1325]: [status] notice: members: 1/539995, 2/422073, 3/1325
Sep 14 17:41:04 node3.mydomain.com pmxcfs[1325]: [status] notice: starting data syncronisation
Sep 14 17:41:04 node3.mydomain.com corosync[8654]:   [QUORUM] This node is within the primary component and will provide service.
Sep 14 17:41:04 node3.mydomain.com corosync[8654]:   [QUORUM] Members[3]: 1 2 3
Sep 14 17:41:04 node3.mydomain.com pmxcfs[1325]: [status] notice: node has quorum
Sep 14 17:41:04 node3.mydomain.com corosync[8654]:   [MAIN  ] Completed service synchronization, ready to provide service.
Sep 14 17:41:04 node3.mydomain.com pmxcfs[1325]: [dcdb] notice: received sync request (epoch 1/539995/0000001C)
Sep 14 17:41:04 node3.mydomain.com pmxcfs[1325]: [status] notice: received sync request (epoch 1/539995/0000001B)
Sep 14 17:41:04 node3.mydomain.com corosync[8654]:   [TOTEM ] Retransmit List: 11 12 13
Sep 14 17:41:04 node3.mydomain.com corosync[8654]:   [TOTEM ] Retransmit List: 11 12 13
Sep 14 17:41:04 node3.mydomain.com corosync[8654]:   [TOTEM ] Retransmit List: 11 12 13
Sep 14 17:41:05 node3.mydomain.com corosync[8654]:   [TOTEM ] Retransmit List: 11 12 13
Sep 14 17:41:05 node3.mydomain.com corosync[8654]:   [TOTEM ] Retransmit List: 11 12 13
Sep 14 17:41:05 node3.mydomain.com corosync[8654]:   [TOTEM ] Retransmit List: 11 12 13
Sep 14 17:41:05 node3.mydomain.com corosync[8654]:   [TOTEM ] Retransmit List: 11 12 13
Sep 14 17:41:06 node3.mydomain.com corosync[8654]:   [TOTEM ] Retransmit List: 11 12 13
Sep 14 17:41:06 node3.mydomain.com corosync[8654]:   [TOTEM ] Retransmit List: 11 12 13
Sep 14 17:41:07 node3.mydomain.com corosync[8654]:   [TOTEM ] Retransmit List: 11 12 13 1f
Sep 14 17:41:07 node3.mydomain.com corosync[8654]:   [TOTEM ] Retransmit List: 11 12 13 20
Sep 14 17:41:08 node3.mydomain.com corosync[8654]:   [TOTEM ] Retransmit List: 11 12 13
Sep 14 17:41:09 node3.mydomain.com corosync[8654]:   [TOTEM ] Retransmit List: 11 12 13
Sep 14 17:41:09 node3.mydomain.com corosync[8654]:   [TOTEM ] Retransmit List: 11 12 13
Sep 14 17:41:09 node3.mydomain.com corosync[8654]:   [TOTEM ] Retransmit List: 11 12 13
Sep 14 17:41:10 node3.mydomain.com corosync[8654]:   [TOTEM ] Retransmit List: 11 12 13
Sep 14 17:41:10 node3.mydomain.com corosync[8654]:   [TOTEM ] Retransmit List: 11 12 13 22
Sep 14 17:41:10 node3.mydomain.com corosync[8654]:   [TOTEM ] Retransmit List: 11 12 13 22
Sep 14 17:41:10 node3.mydomain.com corosync[8654]:   [TOTEM ] Retransmit List: 11 12 13 22
Sep 14 17:41:10 node3.mydomain.com corosync[8654]:   [TOTEM ] Retransmit List: 11 12 13 22
Sep 14 17:41:10 node3.mydomain.com corosync[8654]:   [TOTEM ] Retransmit List: 11 12 13 22
Sep 14 17:41:10 node3.mydomain.com corosync[8654]:   [TOTEM ] Retransmit List: 11 12 13 22
Sep 14 17:41:10 node3.mydomain.com corosync[8654]:   [TOTEM ] Retransmit List: 11 12 13 22
Sep 14 17:41:10 node3.mydomain.com corosync[8654]:   [TOTEM ] Retransmit List: 11 12 13 22
Sep 14 17:41:10 node3.mydomain.com corosync[8654]:   [TOTEM ] Retransmit List: 11 12 13 22
Sep 14 17:41:11 node3.mydomain.com corosync[8654]:   [TOTEM ] Retransmit List: 11 12 13 22