Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
+ [[ -f /env/_master ]]
+ MASTER_IP=10.0.135.159
+ [[ 10.0.221.58 == \1\0\.\0\.\1\3\5\.\1\5\9 ]]
+ echo 'joining cluster at 10.0.135.159'
joining cluster at 10.0.135.159
++ bracketify 10.0.221.58
++ case "$1" in
++ echo 10.0.221.58
++ bracketify 10.0.135.159
++ case "$1" in
++ echo 10.0.135.159
+ exec /usr/share/ovn/scripts/ovn-ctl --db-sb-cluster-local-port=9644 --db-sb-cluster-remote-port=9644 --db-sb-cluster-local-addr=10.0.221.58 --db-sb-cluster-remote-addr=10.0.135.159 --no-monitor --db-sb-cluster-local-proto=ssl --db-sb-cluster-remote-proto=ssl --ovn-sb-db-ssl-key=/ovn-cert/tls.key --ovn-sb-db-ssl-cert=/ovn-cert/tls.crt --ovn-sb-db-ssl-ca-cert=/ovn-ca/ca-bundle.crt '--ovn-sb-log=-vconsole:info -vfile:off' run_sb_ovsdb
2020-06-02T15:29:27Z|00001|vlog|INFO|opened log file /var/log/ovn/ovsdb-server-sb.log
2020-06-02T15:29:28Z|00002|ovsdb_server|INFO|ovsdb-server (Open vSwitch) 2.12.0
2020-06-02T15:29:28Z|00003|reconnect|INFO|ssl:10.0.173.114:9644: connecting...
2020-06-02T15:29:28Z|00004|reconnect|INFO|ssl:10.0.135.159:9644: connecting...
2020-06-02T15:29:28Z|00005|reconnect|INFO|ssl:10.0.173.114:9644: connected
2020-06-02T15:29:28Z|00006|reconnect|INFO|ssl:10.0.135.159:9644: connected
2020-06-02T15:29:28Z|00007|raft|INFO|server 77d9 is leader for term 708
2020-06-02T15:29:28Z|00008|raft|INFO|rejecting append_request because previous entry 708,83425 not in local log (mismatch past end of log)
2020-06-02T15:29:31Z|00009|raft|INFO|ssl:10.0.173.114:59758: learned server ID db04
2020-06-02T15:29:31Z|00010|raft|INFO|ssl:10.0.173.114:59758: learned remote address ssl:10.0.173.114:9644
2020-06-02T15:29:33Z|00011|raft|INFO|ssl:10.0.135.159:53462: learned server ID 77d9
2020-06-02T15:29:33Z|00012|raft|INFO|ssl:10.0.135.159:53462: learned remote address ssl:10.0.135.159:9644
2020-06-02T15:29:37Z|00013|memory|INFO|109752 kB peak resident set size after 10.1 seconds
2020-06-02T15:29:37Z|00014|memory|INFO|cells:92114 monitors:0
2020-06-02T15:49:18Z|00015|memory|INFO|peak resident set size grew 77% in last 1180.4 seconds, from 109752 kB to 194148 kB
2020-06-02T15:49:18Z|00016|memory|INFO|cells:253767 monitors:2 sessions:5
2020-06-02T15:49:58Z|00017|memory|INFO|peak resident set size grew 60% in last 40.3 seconds, from 194148 kB to 310864 kB
2020-06-02T15:49:58Z|00018|memory|INFO|backlog:34 cells:411644 monitors:2 sessions:11 triggers:1
2020-06-02T15:54:18Z|00019|memory|INFO|peak resident set size grew 89% in last 260.3 seconds, from 310864 kB to 587724 kB
2020-06-02T15:54:18Z|00020|memory|INFO|cells:749903 monitors:2 sessions:20
2020-06-02T15:54:32Z|00021|raft|INFO|server db04 is leader for term 709
2020-06-02T15:54:33Z|00022|raft|INFO|rejecting term 708 < current term 709 received in append_request message from server 77d9
2020-06-02T15:54:33Z|00023|raft|INFO|rejecting term 708 < current term 709 received in append_request message from server 77d9
2020-06-02T15:54:36Z|00024|poll_loop|INFO|wakeup due to [POLLIN] on fd 48 (10.0.221.58:9642<->10.0.222.46:60246) at lib/stream-ssl.c:827 (56% CPU usage)
2020-06-02T15:54:36Z|00025|poll_loop|INFO|wakeup due to [POLLOUT] on fd 49 (10.0.221.58:9642<->10.0.203.110:60468) at lib/stream-ssl.c:793 (56% CPU usage)
2020-06-02T15:54:36Z|00026|poll_loop|INFO|wakeup due to 0-ms timeout at lib/stream-ssl.c:829 (56% CPU usage)
2020-06-02T15:54:36Z|00027|poll_loop|INFO|wakeup due to [POLLOUT] on fd 49 (10.0.221.58:9642<->10.0.203.110:60468) at lib/stream-ssl.c:793 (56% CPU usage)
2020-06-02T15:54:36Z|00028|poll_loop|INFO|wakeup due to 0-ms timeout at ovsdb/jsonrpc-server.c:599 (56% CPU usage)
2020-06-02T15:54:36Z|00029|poll_loop|INFO|wakeup due to [POLLIN] on fd 26 (10.0.221.58:9644<->10.0.173.114:59758) at lib/stream-ssl.c:827 (56% CPU usage)
2020-06-02T15:54:36Z|00030|poll_loop|INFO|wakeup due to [POLLIN] on fd 20 (FIFO pipe:[103545371]) at ovsdb/log.c:994 (56% CPU usage)
2020-06-02T15:54:36Z|00031|poll_loop|INFO|wakeup due to [POLLIN] on fd 26 (10.0.221.58:9644<->10.0.173.114:59758) at lib/stream-ssl.c:827 (56% CPU usage)
2020-06-02T15:54:36Z|00032|poll_loop|INFO|wakeup due to 0-ms timeout at ovsdb/jsonrpc-server.c:599 (56% CPU usage)
2020-06-02T15:54:36Z|00033|poll_loop|INFO|wakeup due to [POLLOUT] on fd 49 (10.0.221.58:9642<->10.0.203.110:60468) at lib/stream-ssl.c:793 (56% CPU usage)
2020-06-02T15:54:39Z|00034|raft|INFO|term 710: 1070 ms timeout expired, starting election
2020-06-02T15:54:39Z|00035|raft|INFO|term 710: elected leader by 2+ of 3 servers
2020-06-02T15:54:39Z|00036|raft|INFO|rejecting term 709 < current term 710 received in append_request message from server db04
2020-06-02T15:54:39Z|00037|raft|INFO|rejecting term 709 < current term 710 received in vote_reply message from server db04
2020-06-02T15:54:46Z|00038|raft|INFO|current entry eid 5d73a4e0-ef0c-41ff-95ff-202293c902a9 does not match prerequisite 73ffc06e-e4ea-4a08-8fbc-7417852cc9b9 in execute_command_request
2020-06-02T15:55:04Z|00039|raft|INFO|current entry eid ef1eaed1-341b-46f0-aa2d-0ee664d4e66d does not match prerequisite 6efe8792-1179-4115-b8ef-298e9630d95c in execute_command_request
2020-06-02T15:55:09Z|00040|timeval|WARN|Unreasonably long 1299ms poll interval (1222ms user, 57ms system)
2020-06-02T15:55:09Z|00041|timeval|WARN|faults: 18969 minor, 0 major
2020-06-02T15:55:09Z|00042|timeval|WARN|context switches: 0 voluntary, 4 involuntary
2020-06-02T15:55:09Z|00043|coverage|INFO|Event coverage, avg rate over last: 5 seconds, last minute, last hour, hash=1e253855:
2020-06-02T15:55:09Z|00044|coverage|INFO|hmap_pathological 15.2/sec 17.483/sec 1.5094/sec total: 5675
2020-06-02T15:55:09Z|00045|coverage|INFO|hmap_expand 2910.2/sec 9145.717/sec 430.6833/sec total: 1694180
2020-06-02T15:55:09Z|00046|coverage|INFO|lockfile_lock 0.0/sec 0.000/sec 0.0003/sec total: 1
2020-06-02T15:55:09Z|00047|coverage|INFO|poll_create_node 1165.8/sec 995.967/sec 89.5494/sec total: 328868
2020-06-02T15:55:09Z|00048|coverage|INFO|poll_zero_timeout 17.0/sec 12.583/sec 1.6619/sec total: 6129
2020-06-02T15:55:09Z|00049|coverage|INFO|seq_change 41.2/sec 38.633/sec 5.8661/sec total: 21337
2020-06-02T15:55:09Z|00050|coverage|INFO|pstream_open 0.0/sec 0.000/sec 0.0011/sec total: 4
2020-06-02T15:55:09Z|00051|coverage|INFO|stream_open 0.0/sec 0.000/sec 0.0006/sec total: 2
2020-06-02T15:55:09Z|00052|coverage|INFO|unixctl_received 0.2/sec 0.100/sec 0.0419/sec total: 151
2020-06-02T15:55:09Z|00053|coverage|INFO|unixctl_replied 0.2/sec 0.100/sec 0.0419/sec total: 151
2020-06-02T15:55:09Z|00054|coverage|INFO|util_xalloc 149271.2/sec 438703.967/sec 21563.9425/sec total: 84617205
2020-06-02T15:55:09Z|00055|coverage|INFO|68 events never hit
2020-06-02T15:55:09Z|00056|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T15:55:09Z|00057|jsonrpc|WARN|ssl:10.0.173.114:56332: receive error: Protocol error
2020-06-02T15:55:09Z|00058|raft|WARN|ignoring vote request received as leader
2020-06-02T15:55:09Z|00059|raft|INFO|rejected append_reply (not leader)
2020-06-02T15:55:09Z|00060|raft|INFO|rejected append_reply (not leader)
2020-06-02T15:55:09Z|00061|raft|INFO|rejected append_reply (not leader)
2020-06-02T15:55:09Z|00062|raft|INFO|rejected append_reply (not leader)
2020-06-02T15:55:09Z|00063|raft|INFO|rejected append_reply (not leader)
2020-06-02T15:55:09Z|00064|raft|INFO|rejected append_reply (not leader)
2020-06-02T15:55:09Z|00065|raft|INFO|rejected append_reply (not leader)
2020-06-02T15:55:09Z|00066|raft|INFO|rejected append_reply (not leader)
2020-06-02T15:55:09Z|00067|raft|INFO|rejected append_reply (not leader)
2020-06-02T15:55:09Z|00068|raft|INFO|rejected append_reply (not leader)
2020-06-02T15:55:09Z|00069|raft|INFO|rejecting term 710 < current term 711 received in append_reply message from server 77d9
2020-06-02T15:55:09Z|00070|raft|INFO|rejecting term 710 < current term 711 received in append_reply message from server 77d9
2020-06-02T15:55:09Z|00071|reconnect|WARN|ssl:10.0.173.114:56332: connection dropped (Protocol error)
2020-06-02T15:55:10Z|00072|poll_loop|INFO|Dropped 129 log messages in last 33 seconds (most recently, 30 seconds ago) due to excessive rate
2020-06-02T15:55:10Z|00073|poll_loop|INFO|wakeup due to [POLLOUT] on fd 54 (10.0.221.58:9642<->10.0.218.162:44044) at lib/stream-ssl.c:793 (58% CPU usage)
2020-06-02T15:55:10Z|00074|poll_loop|INFO|wakeup due to [POLLOUT] on fd 54 (10.0.221.58:9642<->10.0.218.162:44044) at lib/stream-ssl.c:793 (58% CPU usage)
2020-06-02T15:55:10Z|00075|poll_loop|INFO|wakeup due to [POLLOUT] on fd 54 (10.0.221.58:9642<->10.0.218.162:44044) at lib/stream-ssl.c:793 (58% CPU usage)
2020-06-02T15:55:10Z|00076|poll_loop|INFO|wakeup due to [POLLOUT] on fd 54 (10.0.221.58:9642<->10.0.218.162:44044) at lib/stream-ssl.c:793 (58% CPU usage)
2020-06-02T15:55:10Z|00077|poll_loop|INFO|wakeup due to [POLLOUT] on fd 54 (10.0.221.58:9642<->10.0.218.162:44044) at lib/stream-ssl.c:793 (58% CPU usage)
2020-06-02T15:55:10Z|00078|raft|WARN|ignoring vote request received after only 616 ms (minimum election time is 1000 ms)
2020-06-02T15:55:11Z|00079|raft|INFO|term 712: 1602 ms timeout expired, starting election
2020-06-02T15:55:11Z|00080|raft|INFO|term 712: elected leader by 2+ of 3 servers
2020-06-02T15:55:13Z|00081|timeval|WARN|Unreasonably long 1129ms poll interval (1048ms user, 46ms system)
2020-06-02T15:55:13Z|00082|timeval|WARN|faults: 10725 minor, 0 major
2020-06-02T15:55:13Z|00083|timeval|WARN|disk: 0 reads, 8 writes
2020-06-02T15:55:13Z|00084|timeval|WARN|context switches: 0 voluntary, 6 involuntary
2020-06-02T15:55:13Z|00085|coverage|INFO|Skipping details of duplicate event coverage for hash=1e253855
2020-06-02T15:55:13Z|00086|poll_loop|INFO|Dropped 185 log messages in last 3 seconds (most recently, 1 seconds ago) due to excessive rate
2020-06-02T15:55:13Z|00087|poll_loop|INFO|wakeup due to [POLLIN] on fd 27 (10.0.221.58:9644<->10.0.135.159:53462) at lib/stream-ssl.c:827 (63% CPU usage)
2020-06-02T15:55:22Z|00088|timeval|WARN|Unreasonably long 2740ms poll interval (2572ms user, 81ms system)
2020-06-02T15:55:22Z|00089|timeval|WARN|faults: 17973 minor, 0 major
2020-06-02T15:55:22Z|00090|timeval|WARN|disk: 0 reads, 78608 writes
2020-06-02T15:55:22Z|00091|timeval|WARN|context switches: 179 voluntary, 20 involuntary
2020-06-02T15:55:22Z|00092|coverage|INFO|Skipping details of duplicate event coverage for hash=1e253855
2020-06-02T15:55:22Z|00093|poll_loop|INFO|Dropped 157 log messages in last 9 seconds (most recently, 6 seconds ago) due to excessive rate
2020-06-02T15:55:22Z|00094|poll_loop|INFO|wakeup due to [POLLIN] on fd 27 (10.0.221.58:9644<->10.0.135.159:53462) at lib/stream-ssl.c:827 (85% CPU usage)
2020-06-02T15:55:22Z|00095|raft|WARN|ignoring vote request received as leader
2020-06-02T15:55:22Z|00096|raft|INFO|server db04 is leader for term 713
2020-06-02T15:55:22Z|00097|raft|INFO|rejected append_reply (not leader)
2020-06-02T15:55:22Z|00098|raft|INFO|rejected append_reply (not leader)
2020-06-02T15:55:23Z|00099|timeval|WARN|Unreasonably long 1309ms poll interval (1277ms user, 19ms system)
2020-06-02T15:55:23Z|00100|timeval|WARN|faults: 227 minor, 0 major
2020-06-02T15:55:23Z|00101|timeval|WARN|context switches: 0 voluntary, 44 involuntary
2020-06-02T15:55:24Z|00102|poll_loop|INFO|Dropped 132 log messages in last 3 seconds (most recently, 0 seconds ago) due to excessive rate
2020-06-02T15:55:24Z|00103|poll_loop|INFO|wakeup due to 133-ms timeout at ovsdb/ovsdb-server.c:279 (85% CPU usage)
2020-06-02T15:55:29Z|00104|timeval|WARN|Unreasonably long 1313ms poll interval (1270ms user, 22ms system)
2020-06-02T15:55:29Z|00105|timeval|WARN|faults: 914 minor, 0 major
2020-06-02T15:55:29Z|00106|timeval|WARN|context switches: 0 voluntary, 2 involuntary
2020-06-02T15:55:33Z|00107|raft|INFO|term 714: 1191 ms timeout expired, starting election
2020-06-02T15:55:33Z|00108|raft|INFO|term 714: elected leader by 2+ of 3 servers
2020-06-02T15:55:33Z|00109|raft|INFO|rejecting term 713 < current term 714 received in append_request message from server db04
2020-06-02T15:55:33Z|00110|raft|INFO|rejecting term 713 < current term 714 received in vote_reply message from server db04
2020-06-02T15:55:37Z|00111|raft|WARN|ignoring vote request received as leader
2020-06-02T15:55:38Z|00112|timeval|WARN|Unreasonably long 2210ms poll interval (2096ms user, 82ms system)
2020-06-02T15:55:38Z|00113|timeval|WARN|faults: 12993 minor, 0 major
2020-06-02T15:55:38Z|00114|timeval|WARN|context switches: 0 voluntary, 3 involuntary
2020-06-02T15:55:38Z|00115|raft|WARN|ignoring vote request received as leader
2020-06-02T15:55:38Z|00116|raft|INFO|rejected append_reply (not leader)
2020-06-02T15:55:38Z|00117|raft|INFO|rejected append_reply (not leader)
2020-06-02T15:55:38Z|00118|raft|INFO|rejected append_reply (not leader)
2020-06-02T15:55:38Z|00119|raft|INFO|rejected append_reply (not leader)
2020-06-02T15:55:38Z|00120|raft|WARN|ignoring vote request received after only 500 ms (minimum election time is 1000 ms)
2020-06-02T15:55:38Z|00121|poll_loop|INFO|Dropped 15 log messages in last 14 seconds (most recently, 14 seconds ago) due to excessive rate
2020-06-02T15:55:38Z|00122|poll_loop|INFO|wakeup due to [POLLOUT] on fd 57 (10.0.221.58:9642<->10.0.243.105:58516) at lib/stream-ssl.c:793 (79% CPU usage)
2020-06-02T15:55:38Z|00123|poll_loop|INFO|wakeup due to [POLLOUT] on fd 60 (10.0.221.58:9642<->10.0.237.192:49500) at lib/stream-ssl.c:793 (79% CPU usage)
2020-06-02T15:55:39Z|00124|raft|INFO|term 716: 1158 ms timeout expired, starting election
2020-06-02T15:55:39Z|00125|raft|INFO|server db04 is leader for term 716
2020-06-02T15:55:39Z|00126|raft|INFO|90d3 truncating 1 entries from end of log
2020-06-02T15:55:40Z|00127|raft|INFO|term 717: 1305 ms timeout expired, starting election
2020-06-02T15:55:40Z|00128|raft|INFO|term 717: elected leader by 2+ of 3 servers
2020-06-02T15:55:41Z|00129|raft|INFO|rejecting term 716 < current term 717 received in append_request message from server db04
2020-06-02T15:55:41Z|00130|raft|INFO|rejecting term 716 < current term 717 received in vote_reply message from server db04
2020-06-02T15:55:42Z|00131|memory|INFO|peak resident set size grew 60% in last 84.0 seconds, from 587724 kB to 942028 kB
2020-06-02T15:55:42Z|00132|memory|INFO|cells:1112581 monitors:2 sessions:39
2020-06-02T15:55:47Z|00133|raft|WARN|ignoring vote request received as leader
2020-06-02T15:55:47Z|00134|raft|INFO|rejected append_reply (not leader)
2020-06-02T15:55:47Z|00135|raft|INFO|rejecting term 717 < current term 718 received in append_reply message from server 77d9
2020-06-02T15:55:47Z|00136|timeval|WARN|Unreasonably long 1554ms poll interval (1525ms user, 15ms system)
2020-06-02T15:55:47Z|00137|timeval|WARN|faults: 430 minor, 0 major
2020-06-02T15:55:47Z|00138|timeval|WARN|disk: 0 reads, 8 writes
2020-06-02T15:55:47Z|00139|timeval|WARN|context switches: 0 voluntary, 6 involuntary
2020-06-02T15:55:47Z|00140|poll_loop|INFO|Dropped 1693 log messages in last 9 seconds (most recently, 7 seconds ago) due to excessive rate
2020-06-02T15:55:47Z|00141|poll_loop|INFO|wakeup due to [POLLIN] on fd 68 (10.0.221.58:9642<->10.0.173.114:32828) at lib/stream-ssl.c:815 (68% CPU usage)
2020-06-02T15:55:48Z|00142|raft|INFO|rejected append_reply (not leader)
2020-06-02T15:55:48Z|00143|raft|INFO|server db04 is leader for term 718
2020-06-02T15:55:48Z|00144|poll_loop|INFO|Dropped 48 log messages in last 1 seconds (most recently, 1 seconds ago) due to excessive rate
2020-06-02T15:55:48Z|00145|poll_loop|INFO|wakeup due to [POLLIN] on fd 66 (10.0.221.58:9642<->10.0.207.102:34624) at lib/stream-ssl.c:827 (68% CPU usage)
2020-06-02T15:55:49Z|00146|raft|INFO|server 77d9 is leader for term 719
2020-06-02T15:55:50Z|00147|raft|INFO|rejecting term 718 < current term 719 received in append_request message from server db04
2020-06-02T15:55:52Z|00148|timeval|WARN|Unreasonably long 1406ms poll interval (1378ms user, 19ms system)
2020-06-02T15:55:52Z|00149|timeval|WARN|faults: 430 minor, 0 major
2020-06-02T15:55:52Z|00150|timeval|WARN|disk: 0 reads, 16 writes
2020-06-02T15:55:52Z|00151|timeval|WARN|context switches: 0 voluntary, 2 involuntary
2020-06-02T15:55:57Z|00152|timeval|WARN|Unreasonably long 1593ms poll interval (1486ms user, 69ms system)
2020-06-02T15:55:57Z|00153|timeval|WARN|faults: 20324 minor, 0 major
2020-06-02T15:55:57Z|00154|timeval|WARN|context switches: 0 voluntary, 6 involuntary
2020-06-02T15:55:57Z|00155|poll_loop|INFO|Dropped 54 log messages in last 8 seconds (most recently, 7 seconds ago) due to excessive rate
2020-06-02T15:55:57Z|00156|poll_loop|INFO|wakeup due to [POLLOUT] on fd 70 (10.0.221.58:9642<->10.0.198.203:36748) at lib/stream-ssl.c:793 (74% CPU usage)
2020-06-02T15:55:59Z|00157|raft|INFO|term 720: 1393 ms timeout expired, starting election
2020-06-02T15:55:59Z|00158|raft|INFO|term 720: elected leader by 2+ of 3 servers
2020-06-02T15:55:59Z|00159|raft|INFO|current entry eid 655e6ddc-dcc2-4a0e-9663-5f5df66b98c1 does not match prerequisite 98ed19a3-6498-4b4d-9afd-b00c4c779bb5 in execute_command_request
2020-06-02T15:55:59Z|00160|raft|INFO|current entry eid 655e6ddc-dcc2-4a0e-9663-5f5df66b98c1 does not match prerequisite 98ed19a3-6498-4b4d-9afd-b00c4c779bb5 in execute_command_request
2020-06-02T15:55:59Z|00161|raft|INFO|current entry eid 655e6ddc-dcc2-4a0e-9663-5f5df66b98c1 does not match prerequisite 98ed19a3-6498-4b4d-9afd-b00c4c779bb5 in execute_command_request
2020-06-02T15:55:59Z|00162|raft|INFO|current entry eid 03726824-3df6-4f47-a9db-bae8a0786c7f does not match prerequisite 655e6ddc-dcc2-4a0e-9663-5f5df66b98c1 in execute_command_request
2020-06-02T15:55:59Z|00163|raft|INFO|current entry eid 03726824-3df6-4f47-a9db-bae8a0786c7f does not match prerequisite 655e6ddc-dcc2-4a0e-9663-5f5df66b98c1 in execute_command_request
2020-06-02T15:55:59Z|00164|raft|INFO|current entry eid e3932842-21da-48e3-8cbd-9bd8bf38560f does not match prerequisite 03726824-3df6-4f47-a9db-bae8a0786c7f in execute_command_request
2020-06-02T15:55:59Z|00165|raft|INFO|current entry eid e3932842-21da-48e3-8cbd-9bd8bf38560f does not match prerequisite 03726824-3df6-4f47-a9db-bae8a0786c7f in execute_command_request
2020-06-02T15:55:59Z|00166|raft|INFO|current entry eid ea51d442-25fb-47f8-90d0-34e15d5204e8 does not match prerequisite e3932842-21da-48e3-8cbd-9bd8bf38560f in execute_command_request
2020-06-02T15:55:59Z|00167|raft|INFO|rejecting term 719 < current term 720 received in append_request message from server 77d9
2020-06-02T15:55:59Z|00168|raft|INFO|rejecting term 719 < current term 720 received in append_request message from server 77d9
2020-06-02T15:56:00Z|00169|raft|WARN|ignoring vote request received as leader
2020-06-02T15:56:00Z|00170|timeval|WARN|Unreasonably long 1596ms poll interval (1529ms user, 32ms system)
2020-06-02T15:56:00Z|00171|timeval|WARN|faults: 19 minor, 0 major
2020-06-02T15:56:00Z|00172|timeval|WARN|context switches: 0 voluntary, 3 involuntary
2020-06-02T15:56:00Z|00173|poll_loop|INFO|Dropped 202 log messages in last 4 seconds (most recently, 2 seconds ago) due to excessive rate
2020-06-02T15:56:00Z|00174|poll_loop|INFO|wakeup due to [POLLIN] on fd 70 (10.0.221.58:9642<->10.0.198.203:36748) at lib/stream-ssl.c:827 (55% CPU usage)
2020-06-02T15:56:00Z|00175|raft|INFO|rejected append_reply (not leader)
2020-06-02T15:56:01Z|00176|raft|INFO|server db04 is leader for term 721
2020-06-02T15:56:01Z|00177|raft|INFO|90d3 truncating 1 entries from end of log
2020-06-02T15:56:01Z|00178|raft|INFO|rejected append_reply (not leader)
2020-06-02T15:56:03Z|00179|raft|INFO|server 77d9 is leader for term 722
2020-06-02T15:56:03Z|00180|raft|INFO|rejecting term 721 < current term 722 received in append_request message from server db04
2020-06-02T15:56:03Z|00181|raft|INFO|rejecting term 721 < current term 722 received in append_request message from server db04
2020-06-02T15:56:05Z|00182|raft|INFO|server db04 is leader for term 723
2020-06-02T15:56:06Z|00183|raft|INFO|rejecting term 722 < current term 723 received in append_request message from server 77d9
2020-06-02T15:56:12Z|00184|timeval|WARN|Unreasonably long 1569ms poll interval (1551ms user, 7ms system)
2020-06-02T15:56:12Z|00185|timeval|WARN|faults: 537 minor, 0 major
2020-06-02T15:56:12Z|00186|timeval|WARN|disk: 0 reads, 24 writes
2020-06-02T15:56:12Z|00187|timeval|WARN|context switches: 0 voluntary, 2 involuntary
2020-06-02T15:56:12Z|00188|coverage|INFO|Dropped 7 log messages in last 48 seconds (most recently, 11 seconds ago) due to excessive rate
2020-06-02T15:56:12Z|00189|coverage|INFO|Skipping details of duplicate event coverage for hash=1e253855
2020-06-02T15:56:13Z|00190|poll_loop|INFO|Dropped 163 log messages in last 12 seconds (most recently, 9 seconds ago) due to excessive rate
2020-06-02T15:56:13Z|00191|poll_loop|INFO|wakeup due to [POLLIN] on fd 20 (FIFO pipe:[103545371]) at ovsdb/log.c:994 (63% CPU usage)
2020-06-02T15:56:13Z|00192|poll_loop|INFO|wakeup due to [POLLIN] on fd 26 (10.0.221.58:9644<->10.0.173.114:59758) at lib/stream-ssl.c:827 (63% CPU usage)
2020-06-02T15:56:16Z|00193|raft|INFO|server 77d9 is leader for term 724
2020-06-02T15:56:18Z|00194|timeval|WARN|Unreasonably long 1619ms poll interval (1566ms user, 25ms system)
2020-06-02T15:56:18Z|00195|timeval|WARN|faults: 20 minor, 0 major
2020-06-02T15:56:18Z|00196|timeval|WARN|disk: 0 reads, 8 writes
2020-06-02T15:56:18Z|00197|timeval|WARN|context switches: 0 voluntary, 8 involuntary
2020-06-02T15:56:19Z|00198|poll_loop|INFO|Dropped 84 log messages in last 6 seconds (most recently, 3 seconds ago) due to excessive rate
2020-06-02T15:56:19Z|00199|poll_loop|INFO|wakeup due to [POLLIN] on fd 27 (10.0.221.58:9644<->10.0.135.159:53462) at lib/stream-ssl.c:827 (55% CPU usage)
2020-06-02T15:56:21Z|00200|timeval|WARN|Unreasonably long 1660ms poll interval (1593ms user, 32ms system)
2020-06-02T15:56:21Z|00201|timeval|WARN|faults: 20 minor, 0 major
2020-06-02T15:56:21Z|00202|timeval|WARN|disk: 0 reads, 16 writes
2020-06-02T15:56:21Z|00203|timeval|WARN|context switches: 0 voluntary, 9 involuntary
2020-06-02T15:56:24Z|00204|poll_loop|INFO|Dropped 223 log messages in last 6 seconds (most recently, 1 seconds ago) due to excessive rate
2020-06-02T15:56:24Z|00205|poll_loop|INFO|wakeup due to [POLLIN] on fd 27 (10.0.221.58:9644<->10.0.135.159:53462) at lib/stream-ssl.c:827 (55% CPU usage)
2020-06-02T15:56:33Z|00206|timeval|WARN|Unreasonably long 1789ms poll interval (1704ms user, 47ms system)
2020-06-02T15:56:33Z|00207|timeval|WARN|faults: 4638 minor, 0 major
2020-06-02T15:56:33Z|00208|timeval|WARN|context switches: 0 voluntary, 10 involuntary
2020-06-02T15:56:34Z|00209|raft|WARN|ignoring vote request received after only 952 ms (minimum election time is 1000 ms)
2020-06-02T15:56:34Z|00210|poll_loop|INFO|Dropped 19 log messages in last 10 seconds (most recently, 10 seconds ago) due to excessive rate
2020-06-02T15:56:34Z|00211|poll_loop|INFO|wakeup due to [POLLIN] on fd 27 (10.0.221.58:9644<->10.0.135.159:53462) at lib/stream-ssl.c:827 (65% CPU usage)
2020-06-02T15:56:35Z|00212|raft|INFO|server db04 is leader for term 726
2020-06-02T15:56:36Z|00213|poll_loop|INFO|Dropped 1673 log messages in last 2 seconds (most recently, 1 seconds ago) due to excessive rate
2020-06-02T15:56:36Z|00214|poll_loop|INFO|wakeup due to [POLLIN] on fd 26 (10.0.221.58:9644<->10.0.173.114:59758) at lib/stream-ssl.c:827 (65% CPU usage)
2020-06-02T15:56:41Z|00215|timeval|WARN|Unreasonably long 1439ms poll interval (1420ms user, 9ms system)
2020-06-02T15:56:41Z|00216|timeval|WARN|faults: 21 minor, 0 major
2020-06-02T15:56:41Z|00217|timeval|WARN|disk: 0 reads, 8 writes
2020-06-02T15:56:41Z|00218|timeval|WARN|context switches: 0 voluntary, 2 involuntary
2020-06-02T15:58:26Z|00219|raft|INFO|term 727: 1152 ms timeout expired, starting election
2020-06-02T15:58:26Z|00220|raft|INFO|term 727: elected leader by 2+ of 3 servers
2020-06-02T15:58:27Z|00221|raft|INFO|rejecting term 726 < current term 727 received in append_request message from server db04
2020-06-02T15:58:27Z|00222|raft|INFO|rejecting term 726 < current term 727 received in vote_reply message from server db04
2020-06-02T15:58:39Z|00223|raft|INFO|term 728: starting election
2020-06-02T15:58:41Z|00224|raft|INFO|term 729: 2106 ms timeout expired, starting election
2020-06-02T15:58:41Z|00225|timeval|WARN|Unreasonably long 2092ms poll interval (1975ms user, 102ms system)
2020-06-02T15:58:41Z|00226|timeval|WARN|faults: 44541 minor, 0 major
2020-06-02T15:58:41Z|00227|timeval|WARN|disk: 0 reads, 8 writes
2020-06-02T15:58:41Z|00228|timeval|WARN|context switches: 0 voluntary, 5 involuntary
2020-06-02T15:58:41Z|00229|coverage|INFO|Dropped 4 log messages in last 144 seconds (most recently, 120 seconds ago) due to excessive rate
2020-06-02T15:58:41Z|00230|coverage|INFO|Skipping details of duplicate event coverage for hash=1e253855
2020-06-02T15:58:41Z|00231|poll_loop|INFO|Dropped 13 log messages in last 125 seconds (most recently, 125 seconds ago) due to excessive rate
2020-06-02T15:58:41Z|00232|poll_loop|INFO|wakeup due to [POLLOUT] on fd 80 (10.0.221.58:9642<->10.0.181.23:59088) at lib/stream-ssl.c:793 (63% CPU usage)
2020-06-02T15:58:41Z|00233|poll_loop|INFO|wakeup due to [POLLIN] on fd 13 (0.0.0.0:9642<->) at lib/stream-ssl.c:963 (63% CPU usage)
2020-06-02T15:58:41Z|00234|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T15:58:41Z|00235|jsonrpc|WARN|ssl:10.0.173.114:45510: receive error: Protocol error
2020-06-02T15:58:41Z|00236|raft|INFO|rejecting term 727 < current term 729 received in append_reply message from server db04
2020-06-02T15:58:41Z|00237|raft|INFO|rejecting term 727 < current term 729 received in append_reply message from server db04
2020-06-02T15:58:41Z|00238|raft|INFO|rejecting term 727 < current term 729 received in append_reply message from server db04
2020-06-02T15:58:41Z|00239|raft|INFO|rejecting term 727 < current term 729 received in append_reply message from server db04
2020-06-02T15:58:41Z|00240|raft|INFO|rejecting term 727 < current term 729 received in vote_reply message from server db04
2020-06-02T15:58:41Z|00241|raft|INFO|rejecting term 727 < current term 729 received in append_reply message from server 77d9
2020-06-02T15:58:41Z|00242|raft|INFO|rejecting term 727 < current term 729 received in append_reply message from server 77d9
2020-06-02T15:58:41Z|00243|raft|INFO|rejecting term 727 < current term 729 received in append_reply message from server 77d9
2020-06-02T15:58:41Z|00244|raft|INFO|rejecting term 727 < current term 729 received in vote_reply message from server 77d9
2020-06-02T15:58:41Z|00245|poll_loop|INFO|wakeup due to [POLLIN] on fd 77 (10.0.221.58:9642<->10.0.201.88:32790) at lib/stream-ssl.c:827 (63% CPU usage)
2020-06-02T15:58:41Z|00246|poll_loop|INFO|wakeup due to [POLLIN] on fd 70 (10.0.221.58:9642<->10.0.198.203:36748) at lib/stream-ssl.c:827 (63% CPU usage)
2020-06-02T15:58:41Z|00247|poll_loop|INFO|wakeup due to [POLLIN] on fd 58 (10.0.221.58:9642<->10.0.242.179:46102) at lib/stream-ssl.c:827 (63% CPU usage)
2020-06-02T15:58:41Z|00248|poll_loop|INFO|wakeup due to [POLLIN] on fd 37 (10.0.221.58:9642<->10.0.243.146:48166) at lib/stream-ssl.c:827 (63% CPU usage)
2020-06-02T15:58:41Z|00249|poll_loop|INFO|wakeup due to [POLLIN] on fd 50 (10.0.221.58:9642<->10.0.200.93:37614) at lib/stream-ssl.c:827 (63% CPU usage)
2020-06-02T15:58:41Z|00250|poll_loop|INFO|wakeup due to [POLLIN] on fd 35 (10.0.221.58:9642<->10.0.241.34:47468) at lib/stream-ssl.c:827 (63% CPU usage)
2020-06-02T15:58:41Z|00251|poll_loop|INFO|wakeup due to [POLLIN] on fd 75 (10.0.221.58:9642<->10.0.196.174:47270) at lib/stream-ssl.c:827 (63% CPU usage)
2020-06-02T15:58:41Z|00252|poll_loop|INFO|wakeup due to [POLLIN] on fd 13 (0.0.0.0:9642<->) at lib/stream-ssl.c:963 (63% CPU usage)
2020-06-02T15:58:41Z|00253|reconnect|WARN|ssl:10.0.173.114:45510: connection dropped (Protocol error)
2020-06-02T15:58:41Z|00254|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T15:58:41Z|00255|jsonrpc|WARN|ssl:10.0.173.114:45550: receive error: Protocol error
2020-06-02T15:58:41Z|00256|raft|INFO|term 729: elected leader by 2+ of 3 servers
2020-06-02T15:58:41Z|00257|reconnect|WARN|ssl:10.0.173.114:45550: connection dropped (Protocol error)
2020-06-02T15:58:44Z|00258|raft|WARN|ignoring vote request received as leader
2020-06-02T15:58:44Z|00259|raft|INFO|server 77d9 is leader for term 730
2020-06-02T15:58:44Z|00260|raft|INFO|rejected append_reply (not leader)
2020-06-02T15:58:44Z|00261|timeval|WARN|Unreasonably long 1982ms poll interval (1937ms user, 23ms system)
2020-06-02T15:58:44Z|00262|timeval|WARN|faults: 766 minor, 0 major
2020-06-02T15:58:44Z|00263|timeval|WARN|disk: 0 reads, 24 writes
2020-06-02T15:58:44Z|00264|timeval|WARN|context switches: 1 voluntary, 2 involuntary
2020-06-02T15:58:44Z|00265|coverage|INFO|Skipping details of duplicate event coverage for hash=1e253855
2020-06-02T15:58:44Z|00266|stream_ssl|WARN|SSL_write: system error (Connection reset by peer)
2020-06-02T15:58:44Z|00267|jsonrpc|WARN|ssl:10.0.213.14:52464: send error: Connection reset by peer
2020-06-02T15:58:44Z|00268|raft|INFO|rejected append_reply (not leader)
2020-06-02T15:58:44Z|00269|reconnect|WARN|ssl:10.0.213.14:52464: connection dropped (Connection reset by peer)
2020-06-02T15:58:57Z|00270|raft|INFO|server db04 is leader for term 731
2020-06-02T15:58:58Z|00271|raft|INFO|rejecting term 730 < current term 731 received in append_request message from server 77d9
2020-06-02T15:59:02Z|00272|raft|WARN|ignoring vote request received after only 0 ms (minimum election time is 1000 ms)
2020-06-02T15:59:02Z|00273|timeval|WARN|Unreasonably long 2502ms poll interval (2351ms user, 105ms system)
2020-06-02T15:59:02Z|00274|timeval|WARN|faults: 34345 minor, 0 major
2020-06-02T15:59:02Z|00275|timeval|WARN|context switches: 0 voluntary, 9 involuntary
2020-06-02T15:59:02Z|00276|poll_loop|INFO|Dropped 394 log messages in last 20 seconds (most recently, 14 seconds ago) due to excessive rate
2020-06-02T15:59:02Z|00277|poll_loop|INFO|wakeup due to [POLLIN] on fd 13 (0.0.0.0:9642<->) at lib/stream-ssl.c:963 (57% CPU usage)
2020-06-02T15:59:02Z|00278|poll_loop|INFO|wakeup due to [POLLIN] on fd 14 (/var/run/ovn/ovnsb_db.ctl<->) at lib/stream-fd.c:274 (57% CPU usage)
2020-06-02T15:59:02Z|00279|poll_loop|INFO|wakeup due to [POLLOUT] on fd 28 (10.0.221.58:9642<->10.0.160.126:45522) at lib/stream-ssl.c:793 (57% CPU usage)
2020-06-02T15:59:02Z|00280|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T15:59:02Z|00281|jsonrpc|WARN|ssl:10.0.221.58:58142: receive error: Protocol error
2020-06-02T15:59:02Z|00282|reconnect|WARN|ssl:10.0.221.58:58142: connection dropped (Protocol error)
2020-06-02T15:59:02Z|00283|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T15:59:02Z|00284|jsonrpc|WARN|ssl:10.0.173.114:47164: receive error: Protocol error
2020-06-02T15:59:02Z|00285|reconnect|WARN|ssl:10.0.173.114:47164: connection dropped (Protocol error)
2020-06-02T15:59:04Z|00286|timeval|WARN|Unreasonably long 2045ms poll interval (1959ms user, 62ms system)
2020-06-02T15:59:04Z|00287|timeval|WARN|faults: 12417 minor, 0 major
2020-06-02T15:59:04Z|00288|timeval|WARN|disk: 0 reads, 16 writes
2020-06-02T15:59:04Z|00289|timeval|WARN|context switches: 0 voluntary, 2 involuntary
2020-06-02T15:59:04Z|00290|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T15:59:04Z|00291|jsonrpc|WARN|ssl:10.0.242.24:49088: receive error: Protocol error
2020-06-02T15:59:04Z|00292|reconnect|WARN|ssl:10.0.242.24:49088: connection dropped (Protocol error)
2020-06-02T15:59:04Z|00293|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T15:59:04Z|00294|jsonrpc|WARN|ssl:10.0.220.111:45054: receive error: Protocol error
2020-06-02T15:59:04Z|00295|reconnect|WARN|ssl:10.0.220.111:45054: connection dropped (Protocol error)
2020-06-02T15:59:04Z|00296|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T15:59:04Z|00297|jsonrpc|WARN|ssl:10.0.217.62:50346: receive error: Protocol error
2020-06-02T15:59:04Z|00298|raft|INFO|term 734: elected leader by 2+ of 3 servers
2020-06-02T15:59:04Z|00299|reconnect|WARN|ssl:10.0.217.62:50346: connection dropped (Protocol error)
2020-06-02T15:59:04Z|00300|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T15:59:04Z|00301|jsonrpc|WARN|ssl:10.0.219.89:45330: receive error: Protocol error
2020-06-02T15:59:04Z|00302|reconnect|WARN|ssl:10.0.219.89:45330: connection dropped (Protocol error)
2020-06-02T15:59:04Z|00303|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T15:59:04Z|00304|reconnect|WARN|ssl:10.0.248.117:54712: connection dropped (Protocol error)
2020-06-02T15:59:04Z|00305|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T15:59:04Z|00306|reconnect|WARN|ssl:10.0.234.201:40374: connection dropped (Protocol error)
2020-06-02T15:59:04Z|00307|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T15:59:04Z|00308|reconnect|WARN|ssl:10.0.194.219:46626: connection dropped (Protocol error)
2020-06-02T15:59:04Z|00309|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T15:59:04Z|00310|reconnect|WARN|ssl:10.0.253.126:50184: connection dropped (Protocol error)
2020-06-02T15:59:04Z|00311|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T15:59:04Z|00312|reconnect|WARN|ssl:10.0.243.94:35854: connection dropped (Protocol error)
2020-06-02T15:59:04Z|00313|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T15:59:04Z|00314|reconnect|WARN|ssl:10.0.241.220:47344: connection dropped (Protocol error)
2020-06-02T15:59:04Z|00315|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T15:59:04Z|00316|reconnect|WARN|ssl:10.0.228.42:55884: connection dropped (Protocol error)
2020-06-02T15:59:04Z|00317|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T15:59:04Z|00318|reconnect|WARN|ssl:10.0.163.107:36486: connection dropped (Protocol error)
2020-06-02T15:59:04Z|00319|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T15:59:04Z|00320|reconnect|WARN|ssl:10.0.245.169:33874: connection dropped (Protocol error)
2020-06-02T15:59:04Z|00321|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T15:59:04Z|00322|reconnect|WARN|ssl:10.0.230.31:37884: connection dropped (Protocol error)
2020-06-02T15:59:04Z|00323|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T15:59:04Z|00324|reconnect|WARN|ssl:10.0.238.156:58852: connection dropped (Protocol error)
2020-06-02T15:59:04Z|00325|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T15:59:04Z|00326|reconnect|WARN|ssl:10.0.239.80:33198: connection dropped (Protocol error)
2020-06-02T15:59:05Z|00327|poll_loop|INFO|Dropped 313 log messages in last 4 seconds (most recently, 0 seconds ago) due to excessive rate
2020-06-02T15:59:05Z|00328|poll_loop|INFO|wakeup due to [POLLOUT] on fd 29 (10.0.221.58:9642<->10.0.172.172:38178) at lib/stream-ssl.c:793 (87% CPU usage)
2020-06-02T15:59:05Z|00329|raft|WARN|ignoring vote request received as leader
2020-06-02T15:59:05Z|00330|raft|INFO|Dropped 7 log messages in last 187 seconds (most recently, 185 seconds ago) due to excessive rate
2020-06-02T15:59:05Z|00331|raft|INFO|current entry eid aa2e5f09-250f-491e-bc10-b065a50e7615 does not match prerequisite b3894abe-c0ee-4a8b-b313-65cd677a1356 in execute_command_request
2020-06-02T15:59:05Z|00332|raft|INFO|current entry eid aa2e5f09-250f-491e-bc10-b065a50e7615 does not match prerequisite b3894abe-c0ee-4a8b-b313-65cd677a1356 in execute_command_request
2020-06-02T15:59:05Z|00333|raft|INFO|current entry eid eddc864d-a7c5-405a-8dcf-1f54d24b1063 does not match prerequisite aa2e5f09-250f-491e-bc10-b065a50e7615 in execute_command_request
2020-06-02T15:59:08Z|00334|raft|INFO|current entry eid 2d149f0f-b9a9-4c15-ba3f-65b0d196a99a does not match prerequisite c8746076-4276-48dd-a3db-3ec3e452d9d9 in execute_command_request
2020-06-02T15:59:08Z|00335|raft|WARN|ignoring vote request received as leader
2020-06-02T15:59:08Z|00336|timeval|WARN|Unreasonably long 2416ms poll interval (2340ms user, 40ms system)
2020-06-02T15:59:08Z|00337|timeval|WARN|faults: 11304 minor, 0 major
2020-06-02T15:59:08Z|00338|timeval|WARN|context switches: 0 voluntary, 6 involuntary
2020-06-02T15:59:08Z|00339|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T15:59:08Z|00340|raft|INFO|rejected append_reply (not leader)
2020-06-02T15:59:08Z|00341|reconnect|WARN|ssl:10.0.214.8:43276: connection dropped (Protocol error)
2020-06-02T15:59:08Z|00342|raft|WARN|ignoring vote request received after only 268 ms (minimum election time is 1000 ms)
2020-06-02T15:59:09Z|00343|raft|INFO|rejected append_reply (not leader)
2020-06-02T15:59:12Z|00344|raft|INFO|server 77d9 is leader for term 737
2020-06-02T15:59:12Z|00345|raft|INFO|90d3 truncating 1 entries from end of log
2020-06-02T15:59:12Z|00346|timeval|WARN|Unreasonably long 2517ms poll interval (2445ms user, 53ms system)
2020-06-02T15:59:12Z|00347|timeval|WARN|faults: 18810 minor, 0 major
2020-06-02T15:59:12Z|00348|timeval|WARN|disk: 0 reads, 40 writes
2020-06-02T15:59:12Z|00349|timeval|WARN|context switches: 1 voluntary, 7 involuntary
2020-06-02T15:59:12Z|00350|coverage|INFO|Dropped 3 log messages in last 11 seconds (most recently, 5 seconds ago) due to excessive rate
2020-06-02T15:59:12Z|00351|coverage|INFO|Skipping details of duplicate event coverage for hash=1e253855
2020-06-02T15:59:12Z|00352|poll_loop|INFO|Dropped 337 log messages in last 7 seconds (most recently, 3 seconds ago) due to excessive rate
2020-06-02T15:59:12Z|00353|poll_loop|INFO|wakeup due to [POLLOUT] on fd 70 (10.0.221.58:9642<->10.0.163.90:47128) at lib/stream-ssl.c:793 (74% CPU usage)
2020-06-02T15:59:14Z|00354|timeval|WARN|Unreasonably long 2077ms poll interval (2028ms user, 34ms system)
2020-06-02T15:59:14Z|00355|timeval|WARN|faults: 1215 minor, 0 major
2020-06-02T15:59:14Z|00356|timeval|WARN|disk: 0 reads, 24 writes
2020-06-02T15:59:14Z|00357|timeval|WARN|context switches: 0 voluntary, 3 involuntary
2020-06-02T15:59:14Z|00358|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T15:59:14Z|00359|jsonrpc|WARN|Dropped 13 log messages in last 10 seconds (most recently, 7 seconds ago) due to excessive rate
2020-06-02T15:59:14Z|00360|jsonrpc|WARN|ssl:10.0.173.114:48096: receive error: Protocol error
2020-06-02T15:59:14Z|00361|reconnect|WARN|ssl:10.0.173.114:48096: connection dropped (Protocol error)
2020-06-02T15:59:14Z|00362|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T15:59:14Z|00363|reconnect|WARN|ssl:10.0.163.107:36558: connection dropped (Protocol error)
2020-06-02T15:59:14Z|00364|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T15:59:14Z|00365|stream_ssl|WARN|SSL_write: system error (Broken pipe)
2020-06-02T15:59:14Z|00366|reconnect|WARN|ssl:10.0.221.58:58254: connection dropped (Broken pipe)
2020-06-02T15:59:14Z|00367|reconnect|WARN|ssl:10.0.248.117:54760: connection dropped (Protocol error)
2020-06-02T15:59:14Z|00368|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T15:59:14Z|00369|reconnect|WARN|ssl:10.0.135.159:38302: connection dropped (Protocol error)
2020-06-02T15:59:14Z|00370|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T15:59:14Z|00371|reconnect|WARN|ssl:10.0.228.42:55914: connection dropped (Protocol error)
2020-06-02T15:59:14Z|00372|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T15:59:14Z|00373|reconnect|WARN|ssl:10.0.253.126:50230: connection dropped (Protocol error)
2020-06-02T15:59:14Z|00374|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T15:59:14Z|00375|reconnect|WARN|ssl:10.0.245.169:33924: connection dropped (Protocol error)
2020-06-02T15:59:14Z|00376|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T15:59:14Z|00377|reconnect|WARN|ssl:10.0.241.220:47392: connection dropped (Protocol error)
2020-06-02T15:59:14Z|00378|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T15:59:14Z|00379|reconnect|WARN|ssl:10.0.220.111:45106: connection dropped (Protocol error)
2020-06-02T15:59:14Z|00380|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T15:59:14Z|00381|reconnect|WARN|ssl:10.0.219.89:45378: connection dropped (Protocol error)
2020-06-02T15:59:19Z|00382|timeval|WARN|Unreasonably long 2159ms poll interval (2062ms user, 54ms system)
2020-06-02T15:59:19Z|00383|timeval|WARN|faults: 29 minor, 0 major
2020-06-02T15:59:19Z|00384|timeval|WARN|disk: 0 reads, 8 writes
2020-06-02T15:59:19Z|00385|timeval|WARN|context switches: 0 voluntary, 16 involuntary
2020-06-02T15:59:19Z|00386|poll_loop|INFO|Dropped 311 log messages in last 6 seconds (most recently, 2 seconds ago) due to excessive rate
2020-06-02T15:59:19Z|00387|poll_loop|INFO|wakeup due to [POLLIN] on fd 13 (0.0.0.0:9642<->) at lib/stream-ssl.c:963 (76% CPU usage)
2020-06-02T15:59:22Z|00388|timeval|WARN|Unreasonably long 2494ms poll interval (2414ms user, 62ms system)
2020-06-02T15:59:22Z|00389|timeval|WARN|faults: 23079 minor, 0 major
2020-06-02T15:59:22Z|00390|timeval|WARN|context switches: 0 voluntary, 5 involuntary
2020-06-02T15:59:22Z|00391|raft|INFO|rejecting term 737 < current term 738 received in append_request message from server 77d9
2020-06-02T15:59:22Z|00392|raft|INFO|rejecting term 737 < current term 738 received in append_request message from server 77d9
2020-06-02T15:59:22Z|00393|raft|INFO|rejecting term 737 < current term 738 received in append_request message from server 77d9
2020-06-02T15:59:22Z|00394|raft|INFO|rejecting term 737 < current term 738 received in append_request message from server 77d9
2020-06-02T15:59:22Z|00395|raft|INFO|rejecting term 737 < current term 738 received in append_request message from server 77d9
2020-06-02T15:59:22Z|00396|raft|INFO|rejecting term 737 < current term 738 received in append_request message from server 77d9
2020-06-02T15:59:22Z|00397|raft|INFO|rejecting term 737 < current term 738 received in append_request message from server 77d9
2020-06-02T15:59:22Z|00398|raft|INFO|rejecting term 737 < current term 738 received in vote_reply message from server db04
2020-06-02T15:59:22Z|00399|raft|INFO|rejecting term 737 < current term 738 received in vote_reply message from server 77d9
2020-06-02T15:59:23Z|00400|raft|ERR|leader for term 738 changed from 77d9 to db04
2020-06-02T15:59:23Z|00401|poll_loop|INFO|Dropped 356 log messages in last 5 seconds (most recently, 1 seconds ago) due to excessive rate
2020-06-02T15:59:23Z|00402|poll_loop|INFO|wakeup due to [POLLIN] on fd 26 (10.0.221.58:9644<->10.0.173.114:59758) at lib/stream-ssl.c:827 (77% CPU usage)
2020-06-02T15:59:30Z|00403|timeval|WARN|Unreasonably long 2444ms poll interval (2335ms user, 73ms system)
2020-06-02T15:59:30Z|00404|timeval|WARN|faults: 15226 minor, 0 major
2020-06-02T15:59:30Z|00405|timeval|WARN|context switches: 0 voluntary, 51 involuntary
2020-06-02T15:59:30Z|00406|poll_loop|INFO|Dropped 6636 log messages in last 7 seconds (most recently, 6 seconds ago) due to excessive rate
2020-06-02T15:59:30Z|00407|poll_loop|INFO|wakeup due to [POLLIN] on fd 13 (0.0.0.0:9642<->) at lib/stream-ssl.c:963 (99% CPU usage)
2020-06-02T15:59:35Z|00408|raft|INFO|rejecting term 738 < current term 739 received in vote_reply message from server 77d9
2020-06-02T15:59:35Z|00409|poll_loop|INFO|Dropped 23931 log messages in last 5 seconds (most recently, 0 seconds ago) due to excessive rate
2020-06-02T15:59:35Z|00410|poll_loop|INFO|wakeup due to 1-ms timeout at ovsdb/trigger.c:164 (97% CPU usage)
2020-06-02T15:59:36Z|00411|raft|INFO|rejecting term 738 < current term 739 received in append_request message from server db04
2020-06-02T15:59:36Z|00412|raft|INFO|rejecting term 738 < current term 739 received in vote_reply message from server db04
2020-06-02T15:59:37Z|00413|raft|INFO|term 740: elected leader by 2+ of 3 servers
2020-06-02T15:59:41Z|00414|raft|INFO|current entry eid 72af119b-65c1-415a-b226-d3bf7f1dbac5 does not match prerequisite 191bffac-7fe3-40f9-9f37-c8c9bee69c75 in execute_command_request
2020-06-02T15:59:43Z|00415|stream_ssl|WARN|SSL_write: system error (Connection reset by peer)
2020-06-02T15:59:43Z|00416|jsonrpc|WARN|Dropped 10 log messages in last 28 seconds (most recently, 28 seconds ago) due to excessive rate
2020-06-02T15:59:43Z|00417|jsonrpc|WARN|ssl:10.0.228.38:59738: send error: Connection reset by peer
2020-06-02T15:59:43Z|00418|stream_ssl|WARN|SSL_write: system error (Connection reset by peer)
2020-06-02T15:59:43Z|00419|jsonrpc|WARN|ssl:10.0.208.195:38900: send error: Connection reset by peer
2020-06-02T15:59:43Z|00420|stream_ssl|WARN|SSL_write: system error (Connection reset by peer)
2020-06-02T15:59:43Z|00421|stream_ssl|WARN|SSL_write: system error (Connection reset by peer)
2020-06-02T15:59:43Z|00422|stream_ssl|WARN|SSL_write: system error (Connection reset by peer)
2020-06-02T15:59:43Z|00423|stream_ssl|WARN|SSL_write: system error (Connection reset by peer)
2020-06-02T15:59:43Z|00424|stream_ssl|WARN|SSL_write: system error (Connection reset by peer)
2020-06-02T15:59:43Z|00425|stream_ssl|WARN|SSL_write: system error (Connection reset by peer)
2020-06-02T15:59:44Z|00426|stream_ssl|WARN|SSL_write: system error (Connection reset by peer)
2020-06-02T15:59:44Z|00427|stream_ssl|WARN|SSL_write: system error (Connection reset by peer)
2020-06-02T15:59:44Z|00428|stream_ssl|WARN|SSL_write: system error (Connection reset by peer)
2020-06-02T15:59:45Z|00429|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T15:59:45Z|00430|timeval|WARN|Unreasonably long 2789ms poll interval (2656ms user, 85ms system)
2020-06-02T15:59:45Z|00431|timeval|WARN|faults: 11127 minor, 0 major
2020-06-02T15:59:45Z|00432|timeval|WARN|disk: 0 reads, 8 writes
2020-06-02T15:59:45Z|00433|timeval|WARN|context switches: 0 voluntary, 9 involuntary
2020-06-02T15:59:45Z|00434|poll_loop|INFO|Dropped 6553 log messages in last 9 seconds (most recently, 6 seconds ago) due to excessive rate
2020-06-02T15:59:45Z|00435|poll_loop|INFO|wakeup due to [POLLIN][POLLOUT][POLLERR][POLLHUP] on fd 77 (10.0.221.58:9642<->) at lib/stream-ssl.c:793 (77% CPU usage)
2020-06-02T15:59:45Z|00436|memory|INFO|peak resident set size grew 51% in last 242.5 seconds, from 942028 kB to 1421132 kB
2020-06-02T15:59:45Z|00437|memory|INFO|backlog:1650739 cells:2065363 monitors:3 sessions:42
2020-06-02T15:59:45Z|00438|stream_ssl|WARN|SSL_write: system error (Connection reset by peer)
2020-06-02T15:59:45Z|00439|stream_ssl|WARN|SSL_write: system error (Connection reset by peer)
2020-06-02T15:59:45Z|00440|stream_ssl|WARN|SSL_write: system error (Connection reset by peer)
2020-06-02T15:59:45Z|00441|reconnect|WARN|ssl:10.0.228.38:59738: connection dropped (Connection reset by peer)
2020-06-02T15:59:45Z|00442|stream_ssl|WARN|SSL_write: system error (Connection reset by peer)
2020-06-02T15:59:45Z|00443|stream_ssl|WARN|SSL_write: system error (Connection reset by peer)
2020-06-02T15:59:45Z|00444|reconnect|WARN|ssl:10.0.208.195:38900: connection dropped (Connection reset by peer)
2020-06-02T15:59:45Z|00445|stream_ssl|WARN|SSL_write: system error (Connection reset by peer)
2020-06-02T15:59:45Z|00446|stream_ssl|WARN|SSL_write: system error (Connection reset by peer)
2020-06-02T15:59:45Z|00447|stream_ssl|WARN|SSL_write: system error (Connection reset by peer)
2020-06-02T15:59:45Z|00448|stream_ssl|WARN|SSL_write: system error (Connection reset by peer)
2020-06-02T15:59:45Z|00449|reconnect|WARN|ssl:10.0.213.193:55356: connection dropped (Connection reset by peer)
2020-06-02T15:59:45Z|00450|reconnect|WARN|ssl:10.0.231.179:57044: connection dropped (Connection reset by peer)
2020-06-02T15:59:45Z|00451|reconnect|WARN|ssl:10.0.233.74:46860: connection dropped (Connection reset by peer)
2020-06-02T15:59:45Z|00452|reconnect|WARN|ssl:10.0.196.174:47958: connection dropped (Connection reset by peer)
2020-06-02T15:59:45Z|00453|reconnect|WARN|ssl:10.0.221.108:34714: connection dropped (Connection reset by peer)
2020-06-02T15:59:45Z|00454|reconnect|WARN|ssl:10.0.251.235:60186: connection dropped (Connection reset by peer)
2020-06-02T15:59:45Z|00455|reconnect|WARN|ssl:10.0.209.243:33114: connection dropped (Connection reset by peer)
2020-06-02T15:59:45Z|00456|reconnect|WARN|ssl:10.0.221.58:58302: connection dropped (Connection reset by peer)
2020-06-02T15:59:45Z|00457|reconnect|WARN|ssl:10.0.204.148:39040: connection dropped (Connection reset by peer)
2020-06-02T15:59:45Z|00458|reconnect|WARN|ssl:10.0.135.159:38778: connection dropped (Protocol error)
2020-06-02T15:59:45Z|00459|raft|WARN|ignoring vote request received as leader
2020-06-02T15:59:48Z|00460|raft|INFO|current entry eid 5872ebd1-d212-45c9-b4ac-bfda7bf9262c does not match prerequisite 72af119b-65c1-415a-b226-d3bf7f1dbac5 in execute_command_request
2020-06-02T15:59:48Z|00461|raft|WARN|ignoring vote request received as leader
2020-06-02T15:59:48Z|00462|raft|INFO|rejected append_reply (not leader)
2020-06-02T15:59:48Z|00463|raft|INFO|rejected append_reply (not leader)
2020-06-02T15:59:48Z|00464|raft|WARN|ignoring vote request received after only 0 ms (minimum election time is 1000 ms)
2020-06-02T15:59:48Z|00465|raft|WARN|ignoring vote request received after only 0 ms (minimum election time is 1000 ms)
2020-06-02T15:59:48Z|00466|timeval|WARN|Unreasonably long 2789ms poll interval (2667ms user, 90ms system)
2020-06-02T15:59:48Z|00467|timeval|WARN|faults: 36765 minor, 0 major
2020-06-02T15:59:48Z|00468|timeval|WARN|disk: 0 reads, 8 writes
2020-06-02T15:59:48Z|00469|timeval|WARN|context switches: 0 voluntary, 4 involuntary
2020-06-02T15:59:48Z|00470|poll_loop|INFO|Dropped 78 log messages in last 4 seconds (most recently, 3 seconds ago) due to excessive rate
2020-06-02T15:59:48Z|00471|poll_loop|INFO|wakeup due to [POLLOUT] on fd 25 (10.0.221.58:9642<->10.0.166.149:46450) at lib/stream-ssl.c:793 (98% CPU usage)
2020-06-02T15:59:48Z|00472|raft|INFO|rejecting term 741 < current term 742 received in append_reply message from server db04
2020-06-02T15:59:48Z|00473|raft|INFO|rejecting term 741 < current term 742 received in append_reply message from server db04
2020-06-02T15:59:48Z|00474|reconnect|WARN|ssl:10.0.221.171:32928: connection dropped (Input/output error)
2020-06-02T15:59:48Z|00475|reconnect|WARN|ssl:10.0.208.133:33684: connection dropped (Input/output error)
2020-06-02T15:59:48Z|00476|reconnect|WARN|ssl:10.0.223.27:44648: connection dropped (Input/output error)
2020-06-02T15:59:48Z|00477|reconnect|WARN|ssl:10.0.192.128:60620: connection dropped (Input/output error)
2020-06-02T15:59:48Z|00478|reconnect|WARN|ssl:10.0.206.170:38430: connection dropped (Input/output error)
2020-06-02T15:59:48Z|00479|reconnect|WARN|ssl:10.0.185.67:36744: connection dropped (Protocol error)
2020-06-02T15:59:48Z|00480|reconnect|WARN|ssl:10.0.191.88:55964: connection dropped (Protocol error)
2020-06-02T15:59:48Z|00481|reconnect|WARN|ssl:10.0.181.23:59332: connection dropped (Protocol error)
2020-06-02T15:59:49Z|00482|raft|WARN|ignoring vote request received after only 962 ms (minimum election time is 1000 ms)
2020-06-02T15:59:50Z|00483|raft|INFO|Dropped 4 log messages in last 45 seconds (most recently, 13 seconds ago) due to excessive rate
2020-06-02T15:59:50Z|00484|raft|INFO|term 743: 1852 ms timeout expired, starting election
2020-06-02T15:59:54Z|00485|raft|INFO|server 77d9 is leader for term 746
2020-06-02T15:59:54Z|00486|raft|INFO|90d3 truncating 1 entries from end of log
2020-06-02T15:59:54Z|00487|timeval|WARN|Unreasonably long 2983ms poll interval (2833ms user, 114ms system)
2020-06-02T15:59:54Z|00488|timeval|WARN|faults: 59261 minor, 0 major
2020-06-02T15:59:54Z|00489|timeval|WARN|disk: 0 reads, 24 writes
2020-06-02T15:59:54Z|00490|timeval|WARN|context switches: 1 voluntary, 15 involuntary
2020-06-02T15:59:54Z|00491|jsonrpc|WARN|Dropped 18 log messages in last 12 seconds (most recently, 6 seconds ago) due to excessive rate
2020-06-02T15:59:54Z|00492|jsonrpc|WARN|unix#179: send error: Broken pipe
2020-06-02T15:59:54Z|00493|stream_ssl|WARN|Dropped 27 log messages in last 10 seconds (most recently, 6 seconds ago) due to excessive rate
2020-06-02T15:59:54Z|00494|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T15:59:54Z|00495|reconnect|WARN|ssl:10.0.160.126:45678: connection dropped (Protocol error)
2020-06-02T15:59:54Z|00496|reconnect|WARN|ssl:10.0.181.159:33128: connection dropped (Protocol error)
2020-06-02T15:59:54Z|00497|reconnect|WARN|ssl:10.0.173.114:52546: connection dropped (Protocol error)
2020-06-02T15:59:54Z|00498|poll_loop|INFO|Dropped 331 log messages in last 6 seconds (most recently, 4 seconds ago) due to excessive rate
2020-06-02T15:59:54Z|00499|poll_loop|INFO|wakeup due to [POLLIN] on fd 13 (0.0.0.0:9642<->) at lib/stream-ssl.c:963 (98% CPU usage)
2020-06-02T15:59:59Z|00500|raft|WARN|Dropped 2 log messages in last 8 seconds (most recently, 3 seconds ago) due to excessive rate
2020-06-02T15:59:59Z|00501|raft|WARN|ignoring vote request received after only 0 ms (minimum election time is 1000 ms)
2020-06-02T15:59:59Z|00502|timeval|WARN|Unreasonably long 3076ms poll interval (2807ms user, 192ms system)
2020-06-02T15:59:59Z|00503|timeval|WARN|faults: 58869 minor, 0 major
2020-06-02T15:59:59Z|00504|timeval|WARN|disk: 0 reads, 8 writes
2020-06-02T15:59:59Z|00505|timeval|WARN|context switches: 0 voluntary, 14 involuntary
2020-06-02T15:59:59Z|00506|stream_ssl|WARN|Dropped 2 log messages in last 4 seconds (most recently, 4 seconds ago) due to excessive rate
2020-06-02T15:59:59Z|00507|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T15:59:59Z|00508|raft|INFO|rejecting term 746 < current term 748 received in append_request message from server 77d9
2020-06-02T15:59:59Z|00509|raft|INFO|90d3 received "prerequisite check failed" reply from 77d9 for unknown command
2020-06-02T15:59:59Z|00510|raft|INFO|90d3 received "prerequisite check failed" reply from 77d9 for unknown command
2020-06-02T15:59:59Z|00511|raft|INFO|90d3 received "prerequisite check failed" reply from 77d9 for unknown command
2020-06-02T15:59:59Z|00512|reconnect|WARN|ssl:10.0.168.135:40112: connection dropped (Protocol error)
2020-06-02T15:59:59Z|00513|reconnect|WARN|ssl:10.0.173.114:53072: connection dropped (Protocol error)
2020-06-02T15:59:59Z|00514|reconnect|WARN|ssl:10.0.184.160:38300: connection dropped (Protocol error)
2020-06-02T15:59:59Z|00515|reconnect|WARN|ssl:10.0.185.138:60988: connection dropped (Protocol error)
2020-06-02T15:59:59Z|00516|reconnect|WARN|ssl:10.0.187.11:51692: connection dropped (Protocol error)
2020-06-02T15:59:59Z|00517|reconnect|WARN|ssl:10.0.187.141:55072: connection dropped (Protocol error)
2020-06-02T15:59:59Z|00518|reconnect|WARN|ssl:10.0.188.1:38942: connection dropped (Protocol error)
2020-06-02T16:00:02Z|00519|raft|INFO|server 77d9 is leader for term 752
2020-06-02T16:00:02Z|00520|raft|INFO|rejecting append_request because previous entry 746,85009 not in local log (mismatch past end of log)
2020-06-02T16:00:02Z|00521|raft|INFO|rejecting append_request because previous entry 746,85009 not in local log (mismatch past end of log)
2020-06-02T16:00:02Z|00522|raft|INFO|rejecting append_request because previous entry 752,85010 not in local log (mismatch past end of log)
2020-06-02T16:00:02Z|00523|raft|INFO|rejecting append_request because previous entry 752,85010 not in local log (mismatch past end of log)
2020-06-02T16:00:02Z|00524|raft|INFO|rejecting append_request because previous entry 752,85010 not in local log (mismatch past end of log)
2020-06-02T16:00:02Z|00525|timeval|WARN|Unreasonably long 3078ms poll interval (2824ms user, 178ms system)
2020-06-02T16:00:02Z|00526|timeval|WARN|faults: 58478 minor, 0 major
2020-06-02T16:00:02Z|00527|timeval|WARN|disk: 0 reads, 16 writes
2020-06-02T16:00:02Z|00528|timeval|WARN|context switches: 0 voluntary, 53 involuntary
2020-06-02T16:00:02Z|00529|poll_loop|INFO|Dropped 145 log messages in last 7 seconds (most recently, 3 seconds ago) due to excessive rate
2020-06-02T16:00:02Z|00530|poll_loop|INFO|wakeup due to [POLLOUT] on fd 37 (10.0.221.58:9642<->10.0.168.255:50706) at lib/stream-ssl.c:793 (97% CPU usage)
2020-06-02T16:00:02Z|00531|jsonrpc|WARN|Dropped 10 log messages in last 7 seconds (most recently, 3 seconds ago) due to excessive rate
2020-06-02T16:00:02Z|00532|jsonrpc|WARN|ssl:10.0.203.69:52724: receive error: Protocol error
2020-06-02T16:00:05Z|00533|reconnect|WARN|ssl:10.0.203.69:52724: connection dropped (Protocol error)
2020-06-02T16:00:05Z|00534|stream_ssl|WARN|Dropped 7 log messages in last 6 seconds (most recently, 3 seconds ago) due to excessive rate
2020-06-02T16:00:05Z|00535|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:00:05Z|00536|timeval|WARN|Unreasonably long 3109ms poll interval (2846ms user, 184ms system)
2020-06-02T16:00:05Z|00537|timeval|WARN|faults: 59109 minor, 0 major
2020-06-02T16:00:05Z|00538|timeval|WARN|disk: 0 reads, 16 writes
2020-06-02T16:00:05Z|00539|timeval|WARN|context switches: 0 voluntary, 19 involuntary
2020-06-02T16:00:05Z|00540|reconnect|WARN|ssl:10.0.218.217:39320: connection dropped (Protocol error)
2020-06-02T16:00:05Z|00541|reconnect|WARN|ssl:10.0.207.102:35828: connection dropped (Protocol error)
2020-06-02T16:00:05Z|00542|reconnect|WARN|ssl:10.0.206.138:48104: connection dropped (Protocol error)
2020-06-02T16:00:05Z|00543|reconnect|WARN|ssl:10.0.251.213:39724: connection dropped (Protocol error)
2020-06-02T16:00:05Z|00544|reconnect|WARN|ssl:10.0.237.106:45382: connection dropped (Protocol error)
2020-06-02T16:00:05Z|00545|reconnect|WARN|ssl:10.0.184.9:47888: connection dropped (Protocol error)
2020-06-02T16:00:05Z|00546|reconnect|WARN|ssl:10.0.198.203:37852: connection dropped (Protocol error)
2020-06-02T16:00:05Z|00547|reconnect|WARN|ssl:10.0.249.214:42576: connection dropped (Protocol error)
2020-06-02T16:00:05Z|00548|reconnect|WARN|ssl:10.0.243.129:47560: connection dropped (Protocol error)
2020-06-02T16:00:05Z|00549|reconnect|WARN|ssl:10.0.221.58:58752: connection dropped (Protocol error)
2020-06-02T16:00:05Z|00550|reconnect|WARN|ssl:10.0.173.114:53722: connection dropped (Protocol error)
2020-06-02T16:00:05Z|00551|poll_loop|INFO|Dropped 46 log messages in last 4 seconds (most recently, 0 seconds ago) due to excessive rate
2020-06-02T16:00:05Z|00552|poll_loop|INFO|wakeup due to [POLLIN] on fd 13 (0.0.0.0:9642<->) at lib/stream-ssl.c:963 (97% CPU usage)
2020-06-02T16:00:08Z|00553|reconnect|WARN|ssl:10.0.253.58:37094: connection dropped (Protocol error)
2020-06-02T16:00:08Z|00554|raft|INFO|rejecting term 752 < current term 754 received in vote_reply message from server db04
2020-06-02T16:00:08Z|00555|raft|INFO|rejecting term 752 < current term 754 received in vote_reply message from server db04
2020-06-02T16:00:08Z|00556|stream_ssl|WARN|Dropped 11 log messages in last 4 seconds (most recently, 1 seconds ago) due to excessive rate
2020-06-02T16:00:08Z|00557|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:00:08Z|00558|reconnect|WARN|ssl:10.0.208.175:46376: connection dropped (Protocol error)
2020-06-02T16:00:11Z|00559|raft|INFO|rejecting term 752 < current term 755 received in append_request message from server 77d9
2020-06-02T16:00:11Z|00560|raft|INFO|rejecting term 752 < current term 755 received in vote_reply message from server 77d9
2020-06-02T16:00:11Z|00561|raft|INFO|rejecting term 752 < current term 755 received in vote_reply message from server 77d9
2020-06-02T16:00:11Z|00562|reconnect|WARN|ssl:10.0.135.159:53462: connection dropped (Protocol error)
2020-06-02T16:00:12Z|00563|raft|INFO|ssl:10.0.135.159:50000: learned server ID 77d9
2020-06-02T16:00:12Z|00564|raft|INFO|ssl:10.0.135.159:50000: learned remote address ssl:10.0.135.159:9644
2020-06-02T16:00:12Z|00565|raft|INFO|rejecting term 756 < current term 757 received in vote_request message from server 77d9
2020-06-02T16:00:13Z|00566|raft|INFO|server db04 is leader for term 758
2020-06-02T16:00:13Z|00567|raft|INFO|rejecting append_request because previous entry 752,85021 not in local log (mismatch past end of log)
2020-06-02T16:00:13Z|00568|raft|INFO|rejecting append_request because previous entry 752,85021 not in local log (mismatch past end of log)
2020-06-02T16:00:16Z|00569|raft|WARN|Dropped 3 log messages in last 15 seconds (most recently, 15 seconds ago) due to excessive rate
2020-06-02T16:00:16Z|00570|raft|WARN|ignoring vote request received after only 3 ms (minimum election time is 1000 ms)
2020-06-02T16:00:16Z|00571|timeval|WARN|Unreasonably long 2436ms poll interval (2355ms user, 65ms system)
2020-06-02T16:00:16Z|00572|timeval|WARN|faults: 20110 minor, 0 major
2020-06-02T16:00:16Z|00573|timeval|WARN|disk: 0 reads, 16 writes
2020-06-02T16:00:16Z|00574|timeval|WARN|context switches: 0 voluntary, 3 involuntary
2020-06-02T16:00:16Z|00575|coverage|INFO|Dropped 10 log messages in last 62 seconds (most recently, 12 seconds ago) due to excessive rate
2020-06-02T16:00:16Z|00576|coverage|INFO|Skipping details of duplicate event coverage for hash=1e253855
2020-06-02T16:00:16Z|00577|poll_loop|INFO|Dropped 730 log messages in last 11 seconds (most recently, 6 seconds ago) due to excessive rate
2020-06-02T16:00:16Z|00578|poll_loop|INFO|wakeup due to [POLLOUT] on fd 27 (10.0.221.58:9642<->10.0.185.193:50476) at lib/stream-ssl.c:793 (63% CPU usage)
2020-06-02T16:00:16Z|00579|raft|INFO|rejecting term 758 < current term 759 received in append_request message from server db04
2020-06-02T16:00:16Z|00580|raft|INFO|rejecting term 758 < current term 759 received in append_request message from server db04
2020-06-02T16:00:16Z|00581|raft|INFO|rejecting term 758 < current term 759 received in append_request message from server db04
2020-06-02T16:00:16Z|00582|raft|INFO|90d3 received "prerequisite check failed" reply from db04 for unknown command
2020-06-02T16:00:16Z|00583|raft|INFO|rejecting term 758 < current term 759 received in append_request message from server db04
2020-06-02T16:00:19Z|00584|raft|INFO|server db04 is leader for term 762
2020-06-02T16:00:19Z|00585|raft|INFO|rejecting append_request because previous entry 758,85023 not in local log (mismatch past end of log)
2020-06-02T16:00:19Z|00586|raft|INFO|rejecting append_request because previous entry 758,85023 not in local log (mismatch past end of log)
2020-06-02T16:00:19Z|00587|raft|INFO|rejecting append_request because previous entry 762,85024 not in local log (mismatch past end of log)
2020-06-02T16:00:19Z|00588|raft|INFO|rejecting append_request because previous entry 762,85024 not in local log (mismatch past end of log)
2020-06-02T16:00:19Z|00589|raft|INFO|rejecting append_request because previous entry 762,85025 not in local log (mismatch past end of log)
2020-06-02T16:00:19Z|00590|raft|INFO|rejecting append_request because previous entry 762,85025 not in local log (mismatch past end of log)
2020-06-02T16:00:19Z|00591|raft|INFO|rejecting append_request because previous entry 762,85026 not in local log (mismatch past end of log)
2020-06-02T16:00:19Z|00592|raft|INFO|rejecting append_request because previous entry 762,85026 not in local log (mismatch past end of log)
2020-06-02T16:00:19Z|00593|raft|INFO|rejecting append_request because previous entry 762,85027 not in local log (mismatch past end of log)
2020-06-02T16:00:19Z|00594|raft|INFO|rejecting append_request because previous entry 762,85027 not in local log (mismatch past end of log)
2020-06-02T16:00:19Z|00595|raft|INFO|rejecting append_request because previous entry 762,85028 not in local log (mismatch past end of log)
2020-06-02T16:00:19Z|00596|raft|INFO|rejecting append_request because previous entry 762,85028 not in local log (mismatch past end of log)
2020-06-02T16:00:19Z|00597|raft|INFO|rejecting append_request because previous entry 762,85029 not in local log (mismatch past end of log)
2020-06-02T16:00:19Z|00598|raft|INFO|rejecting append_request because previous entry 762,85029 not in local log (mismatch past end of log)
2020-06-02T16:00:19Z|00599|raft|INFO|rejecting append_request because previous entry 762,85030 not in local log (mismatch past end of log)
2020-06-02T16:00:19Z|00600|raft|INFO|rejecting append_request because previous entry 762,85030 not in local log (mismatch past end of log)
2020-06-02T16:00:19Z|00601|raft|INFO|rejecting append_request because previous entry 762,85031 not in local log (mismatch past end of log)
2020-06-02T16:00:19Z|00602|raft|INFO|rejecting append_request because previous entry 762,85031 not in local log (mismatch past end of log)
2020-06-02T16:00:19Z|00603|raft|INFO|rejecting append_request because previous entry 762,85032 not in local log (mismatch past end of log)
2020-06-02T16:00:19Z|00604|raft|INFO|rejecting append_request because previous entry 762,85032 not in local log (mismatch past end of log)
2020-06-02T16:00:19Z|00605|raft|INFO|rejecting append_request because previous entry 762,85033 not in local log (mismatch past end of log)
2020-06-02T16:00:19Z|00606|raft|INFO|rejecting append_request because previous entry 762,85033 not in local log (mismatch past end of log)
2020-06-02T16:00:19Z|00607|raft|INFO|rejecting append_request because previous entry 762,85034 not in local log (mismatch past end of log)
2020-06-02T16:00:19Z|00608|raft|INFO|rejecting append_request because previous entry 762,85034 not in local log (mismatch past end of log)
2020-06-02T16:00:19Z|00609|raft|INFO|rejecting append_request because previous entry 762,85035 not in local log (mismatch past end of log)
2020-06-02T16:00:19Z|00610|raft|INFO|rejecting append_request because previous entry 762,85035 not in local log (mismatch past end of log)
2020-06-02T16:00:19Z|00611|raft|INFO|rejecting append_request because previous entry 762,85036 not in local log (mismatch past end of log)
2020-06-02T16:00:19Z|00612|timeval|WARN|Unreasonably long 2873ms poll interval (2779ms user, 74ms system)
2020-06-02T16:00:19Z|00613|timeval|WARN|faults: 30764 minor, 0 major
2020-06-02T16:00:19Z|00614|timeval|WARN|disk: 0 reads, 16 writes
2020-06-02T16:00:19Z|00615|timeval|WARN|context switches: 0 voluntary, 4 involuntary
2020-06-02T16:00:19Z|00616|poll_loop|INFO|Dropped 11 log messages in last 3 seconds (most recently, 3 seconds ago) due to excessive rate
2020-06-02T16:00:19Z|00617|poll_loop|INFO|wakeup due to [POLLOUT] on fd 27 (10.0.221.58:9642<->10.0.185.193:50476) at lib/stream-ssl.c:793 (63% CPU usage)
2020-06-02T16:00:19Z|00618|stream_ssl|WARN|Dropped 1 log messages in last 9 seconds (most recently, 9 seconds ago) due to excessive rate
2020-06-02T16:00:19Z|00619|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:00:19Z|00620|jsonrpc|WARN|Dropped 14 log messages in last 15 seconds (most recently, 9 seconds ago) due to excessive rate
2020-06-02T16:00:19Z|00621|jsonrpc|WARN|ssl:10.0.188.0:50560: receive error: Protocol error
2020-06-02T16:00:19Z|00622|reconnect|WARN|ssl:10.0.188.0:50560: connection dropped (Protocol error)
2020-06-02T16:00:22Z|00623|timeval|WARN|Unreasonably long 2839ms poll interval (2735ms user, 84ms system)
2020-06-02T16:00:22Z|00624|timeval|WARN|faults: 38147 minor, 0 major
2020-06-02T16:00:22Z|00625|timeval|WARN|disk: 0 reads, 8 writes
2020-06-02T16:00:22Z|00626|timeval|WARN|context switches: 0 voluntary, 5 involuntary
2020-06-02T16:00:25Z|00627|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:00:25Z|00628|raft|INFO|rejecting term 762 < current term 763 received in vote_reply message from server 77d9
2020-06-02T16:00:25Z|00629|timeval|WARN|Unreasonably long 2781ms poll interval (2715ms user, 46ms system)
2020-06-02T16:00:25Z|00630|timeval|WARN|faults: 11818 minor, 0 major
2020-06-02T16:00:25Z|00631|timeval|WARN|disk: 0 reads, 8 writes
2020-06-02T16:00:25Z|00632|timeval|WARN|context switches: 0 voluntary, 10 involuntary
2020-06-02T16:00:25Z|00633|poll_loop|INFO|Dropped 25 log messages in last 5 seconds (most recently, 3 seconds ago) due to excessive rate
2020-06-02T16:00:25Z|00634|poll_loop|INFO|wakeup due to [POLLOUT] on fd 27 (10.0.221.58:9642<->10.0.185.193:50476) at lib/stream-ssl.c:793 (100% CPU usage)
2020-06-02T16:00:25Z|00635|reconnect|WARN|ssl:10.0.213.14:54128: connection dropped (Protocol error)
2020-06-02T16:00:25Z|00636|reconnect|WARN|ssl:10.0.173.114:55468: connection dropped (Protocol error)
2020-06-02T16:00:25Z|00637|reconnect|WARN|ssl:10.0.167.40:42680: connection dropped (Protocol error)
2020-06-02T16:00:25Z|00638|raft|INFO|rejecting term 762 < current term 764 received in vote_reply message from server 77d9
2020-06-02T16:00:25Z|00639|reconnect|WARN|ssl:10.0.166.52:41698: connection dropped (Protocol error)
2020-06-02T16:00:25Z|00640|reconnect|WARN|ssl:10.0.202.136:43362: connection dropped (Protocol error)
2020-06-02T16:00:26Z|00641|raft|INFO|rejecting term 763 < current term 764 received in vote_request message from server 77d9
2020-06-02T16:00:28Z|00642|raft|INFO|rejecting term 762 < current term 765 received in append_request message from server db04
2020-06-02T16:00:28Z|00643|raft|INFO|rejecting term 762 < current term 765 received in append_request message from server db04
2020-06-02T16:00:28Z|00644|raft|INFO|rejecting term 762 < current term 765 received in append_request message from server db04
2020-06-02T16:00:28Z|00645|raft|INFO|rejecting term 762 < current term 765 received in append_request message from server db04
2020-06-02T16:00:28Z|00646|raft|INFO|rejecting term 762 < current term 765 received in append_request message from server db04
2020-06-02T16:00:28Z|00647|raft|INFO|rejecting term 762 < current term 765 received in append_request message from server db04
2020-06-02T16:00:28Z|00648|raft|INFO|rejecting term 762 < current term 765 received in append_request message from server db04
2020-06-02T16:00:28Z|00649|raft|INFO|rejecting term 762 < current term 765 received in append_request message from server db04
2020-06-02T16:00:28Z|00650|raft|INFO|rejecting term 762 < current term 765 received in append_request message from server db04
2020-06-02T16:00:28Z|00651|raft|INFO|rejecting term 762 < current term 765 received in append_request message from server db04
2020-06-02T16:00:28Z|00652|raft|INFO|rejecting term 762 < current term 765 received in append_request message from server db04
2020-06-02T16:00:28Z|00653|raft|INFO|rejecting term 762 < current term 765 received in append_request message from server db04
2020-06-02T16:00:28Z|00654|raft|INFO|rejecting term 762 < current term 765 received in append_request message from server db04
2020-06-02T16:00:28Z|00655|raft|INFO|rejecting term 762 < current term 765 received in append_request message from server db04
2020-06-02T16:00:28Z|00656|raft|INFO|rejecting term 762 < current term 765 received in append_request message from server db04
2020-06-02T16:00:28Z|00657|raft|INFO|90d3 received "prerequisite check failed" reply from db04 for unknown command
2020-06-02T16:00:28Z|00658|raft|INFO|rejecting term 762 < current term 765 received in append_request message from server db04
2020-06-02T16:00:28Z|00659|raft|INFO|rejecting term 762 < current term 765 received in append_request message from server db04
2020-06-02T16:00:28Z|00660|raft|INFO|rejecting term 762 < current term 765 received in append_request message from server db04
2020-06-02T16:00:28Z|00661|raft|INFO|rejecting term 762 < current term 765 received in append_request message from server db04
2020-06-02T16:00:28Z|00662|raft|INFO|rejecting term 762 < current term 765 received in append_request message from server db04
2020-06-02T16:00:28Z|00663|raft|INFO|rejecting term 762 < current term 765 received in append_request message from server db04
2020-06-02T16:00:28Z|00664|raft|INFO|rejecting term 762 < current term 765 received in append_request message from server db04
2020-06-02T16:00:28Z|00665|raft|INFO|rejecting term 762 < current term 765 received in append_request message from server db04
2020-06-02T16:00:28Z|00666|raft|INFO|rejecting term 762 < current term 765 received in append_request message from server db04
2020-06-02T16:00:28Z|00667|raft|INFO|rejecting term 762 < current term 765 received in append_request message from server db04
2020-06-02T16:00:28Z|00668|raft|INFO|rejecting term 762 < current term 765 received in append_request message from server db04
2020-06-02T16:00:28Z|00669|raft|INFO|rejecting term 762 < current term 765 received in append_request message from server db04
2020-06-02T16:00:28Z|00670|raft|INFO|rejecting term 762 < current term 765 received in append_request message from server db04
2020-06-02T16:00:28Z|00671|raft|INFO|rejecting term 762 < current term 765 received in append_request message from server db04
2020-06-02T16:00:28Z|00672|raft|INFO|rejecting term 762 < current term 765 received in append_request message from server db04
2020-06-02T16:00:28Z|00673|raft|INFO|rejecting term 762 < current term 765 received in append_request message from server db04
2020-06-02T16:00:28Z|00674|raft|INFO|rejecting term 762 < current term 765 received in append_request message from server db04
2020-06-02T16:00:28Z|00675|raft|INFO|rejecting term 762 < current term 765 received in append_request message from server db04
2020-06-02T16:00:28Z|00676|raft|INFO|rejecting term 762 < current term 765 received in append_request message from server db04
2020-06-02T16:00:28Z|00677|raft|INFO|rejecting term 762 < current term 765 received in append_request message from server db04
2020-06-02T16:00:28Z|00678|raft|INFO|rejecting term 762 < current term 765 received in append_request message from server db04
2020-06-02T16:00:28Z|00679|raft|INFO|rejecting term 762 < current term 765 received in append_request message from server db04
2020-06-02T16:00:28Z|00680|raft|INFO|rejecting term 762 < current term 765 received in append_request message from server db04
2020-06-02T16:00:28Z|00681|raft|INFO|rejecting term 762 < current term 765 received in append_request message from server db04
2020-06-02T16:00:28Z|00682|raft|INFO|rejecting term 762 < current term 765 received in append_request message from server db04
2020-06-02T16:00:28Z|00683|raft|INFO|rejecting term 762 < current term 765 received in vote_reply message from server db04
2020-06-02T16:00:28Z|00684|raft|INFO|rejecting term 762 < current term 765 received in append_request message from server db04
2020-06-02T16:00:28Z|00685|raft|INFO|rejecting term 762 < current term 765 received in append_request message from server db04
2020-06-02T16:00:28Z|00686|raft|INFO|rejecting term 762 < current term 765 received in append_request message from server db04
2020-06-02T16:00:28Z|00687|raft|INFO|rejecting term 762 < current term 765 received in append_request message from server db04
2020-06-02T16:00:28Z|00688|raft|INFO|rejecting term 762 < current term 765 received in append_request message from server db04
2020-06-02T16:00:28Z|00689|raft|INFO|rejecting term 762 < current term 765 received in append_request message from server db04
2020-06-02T16:00:28Z|00690|raft|INFO|rejecting term 762 < current term 765 received in append_request message from server db04
2020-06-02T16:00:28Z|00691|raft|INFO|rejecting term 762 < current term 765 received in append_request message from server db04
2020-06-02T16:00:28Z|00692|raft|INFO|rejecting term 762 < current term 765 received in append_request message from server db04
2020-06-02T16:00:28Z|00693|raft|INFO|rejecting term 762 < current term 765 received in append_request message from server db04
2020-06-02T16:00:28Z|00694|raft|INFO|rejecting term 762 < current term 765 received in append_request message from server db04
2020-06-02T16:00:28Z|00695|raft|INFO|rejecting term 762 < current term 765 received in vote_reply message from server db04
2020-06-02T16:00:28Z|00696|raft|INFO|rejecting term 762 < current term 765 received in vote_reply message from server db04
2020-06-02T16:00:28Z|00697|raft|INFO|rejecting term 762 < current term 765 received in append_request message from server db04
2020-06-02T16:00:30Z|00698|poll_loop|INFO|Dropped 23626 log messages in last 5 seconds (most recently, 0 seconds ago) due to excessive rate
2020-06-02T16:00:30Z|00699|poll_loop|INFO|wakeup due to [POLLIN] on fd 29 (10.0.221.58:9642<->10.0.191.88:56054) at lib/stream-ssl.c:827 (68% CPU usage)
2020-06-02T16:00:31Z|00700|raft|INFO|server 77d9 is leader for term 768
2020-06-02T16:00:31Z|00701|raft|INFO|rejecting append_request because previous entry 762,85053 not in local log (mismatch past end of log)
2020-06-02T16:00:31Z|00702|raft|INFO|rejecting append_request because previous entry 762,85053 not in local log (mismatch past end of log)
2020-06-02T16:00:31Z|00703|raft|INFO|rejecting append_request because previous entry 768,85054 not in local log (mismatch past end of log)
2020-06-02T16:00:32Z|00704|raft|WARN|Dropped 1 log messages in last 13 seconds (most recently, 13 seconds ago) due to excessive rate
2020-06-02T16:00:32Z|00705|raft|WARN|ignoring vote request received after only 329 ms (minimum election time is 1000 ms)
2020-06-02T16:00:40Z|00706|timeval|WARN|Unreasonably long 2796ms poll interval (2679ms user, 89ms system)
2020-06-02T16:00:40Z|00707|timeval|WARN|faults: 26762 minor, 0 major
2020-06-02T16:00:40Z|00708|timeval|WARN|context switches: 0 voluntary, 4 involuntary
2020-06-02T16:00:40Z|00709|poll_loop|INFO|Dropped 153 log messages in last 10 seconds (most recently, 9 seconds ago) due to excessive rate
2020-06-02T16:00:40Z|00710|poll_loop|INFO|wakeup due to [POLLIN] on fd 13 (0.0.0.0:9642<->) at lib/stream-ssl.c:963 (54% CPU usage)
2020-06-02T16:00:40Z|00711|raft|WARN|ignoring vote request received after only 3 ms (minimum election time is 1000 ms)
2020-06-02T16:00:41Z|00712|raft|INFO|rejecting term 768 < current term 770 received in append_request message from server 77d9
2020-06-02T16:00:41Z|00713|raft|INFO|server db04 is leader for term 770
2020-06-02T16:00:42Z|00714|poll_loop|INFO|Dropped 312 log messages in last 2 seconds (most recently, 0 seconds ago) due to excessive rate
2020-06-02T16:00:42Z|00715|poll_loop|INFO|wakeup due to [POLLIN] on fd 26 (10.0.221.58:9644<->10.0.173.114:59758) at lib/stream-ssl.c:827 (54% CPU usage)
2020-06-02T16:00:47Z|00716|timeval|WARN|Unreasonably long 2831ms poll interval (2706ms user, 91ms system)
2020-06-02T16:00:47Z|00717|timeval|WARN|faults: 37019 minor, 0 major
2020-06-02T16:00:47Z|00718|timeval|WARN|context switches: 0 voluntary, 5 involuntary
2020-06-02T16:00:47Z|00719|stream_ssl|WARN|Dropped 4 log messages in last 22 seconds (most recently, 22 seconds ago) due to excessive rate
2020-06-02T16:00:47Z|00720|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:00:47Z|00721|jsonrpc|WARN|Dropped 5 log messages in last 22 seconds (most recently, 22 seconds ago) due to excessive rate
2020-06-02T16:00:47Z|00722|jsonrpc|WARN|ssl:10.0.183.243:48154: receive error: Protocol error
2020-06-02T16:00:47Z|00723|reconnect|WARN|ssl:10.0.183.243:48154: connection dropped (Protocol error)
2020-06-02T16:00:47Z|00724|poll_loop|INFO|Dropped 129 log messages in last 6 seconds (most recently, 0 seconds ago) due to excessive rate
2020-06-02T16:00:47Z|00725|poll_loop|INFO|wakeup due to [POLLOUT] on fd 44 (10.0.221.58:9642<->10.0.182.146:50984) at lib/stream-ssl.c:793 (66% CPU usage)
2020-06-02T16:00:48Z|00726|raft|WARN|ignoring vote request received after only 605 ms (minimum election time is 1000 ms)
2020-06-02T16:00:48Z|00727|raft|INFO|Dropped 10 log messages in last 41 seconds (most recently, 18 seconds ago) due to excessive rate
2020-06-02T16:00:48Z|00728|raft|INFO|term 771: 1037 ms timeout expired, starting election
2020-06-02T16:00:49Z|00729|raft|INFO|rejecting term 770 < current term 771 received in append_request message from server db04
2020-06-02T16:00:49Z|00730|raft|INFO|term 771: elected leader by 2+ of 3 servers
2020-06-02T16:01:11Z|00731|raft|WARN|ignoring vote request received as leader
2020-06-02T16:01:11Z|00732|timeval|WARN|Unreasonably long 3712ms poll interval (3571ms user, 114ms system)
2020-06-02T16:01:11Z|00733|timeval|WARN|faults: 25605 minor, 0 major
2020-06-02T16:01:11Z|00734|timeval|WARN|context switches: 0 voluntary, 5 involuntary
2020-06-02T16:01:11Z|00735|coverage|INFO|Dropped 5 log messages in last 51 seconds (most recently, 24 seconds ago) due to excessive rate
2020-06-02T16:01:11Z|00736|coverage|INFO|Skipping details of duplicate event coverage for hash=1e253855
2020-06-02T16:01:11Z|00737|poll_loop|INFO|Dropped 298 log messages in last 23 seconds (most recently, 21 seconds ago) due to excessive rate
2020-06-02T16:01:11Z|00738|poll_loop|INFO|wakeup due to [POLLOUT] on fd 77 (10.0.221.58:9642<->10.0.254.25:57982) at lib/stream-ssl.c:793 (97% CPU usage)
2020-06-02T16:01:11Z|00739|poll_loop|INFO|wakeup due to [POLLOUT] on fd 70 (10.0.221.58:9642<->10.0.187.141:55272) at lib/stream-ssl.c:793 (97% CPU usage)
2020-06-02T16:01:11Z|00740|poll_loop|INFO|wakeup due to [POLLOUT] on fd 59 (10.0.221.58:9642<->10.0.243.54:47020) at lib/stream-ssl.c:793 (97% CPU usage)
2020-06-02T16:01:11Z|00741|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:01:11Z|00742|jsonrpc|WARN|ssl:10.0.173.114:59942: receive error: Protocol error
2020-06-02T16:01:11Z|00743|raft|WARN|ignoring vote request received as leader
2020-06-02T16:01:11Z|00744|raft|INFO|rejected append_reply (not leader)
2020-06-02T16:01:11Z|00745|reconnect|WARN|ssl:10.0.173.114:59942: connection dropped (Protocol error)
2020-06-02T16:01:11Z|00746|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:01:11Z|00747|jsonrpc|WARN|ssl:10.0.173.114:60004: receive error: Protocol error
2020-06-02T16:01:11Z|00748|reconnect|WARN|ssl:10.0.173.114:60004: connection dropped (Protocol error)
2020-06-02T16:01:11Z|00749|raft|INFO|rejecting term 771 < current term 773 received in append_reply message from server 77d9
2020-06-02T16:01:11Z|00750|raft|INFO|rejecting term 771 < current term 773 received in append_reply message from server 77d9
2020-06-02T16:01:11Z|00751|raft|INFO|rejecting term 772 < current term 773 received in append_reply message from server 77d9
2020-06-02T16:01:11Z|00752|poll_loop|INFO|Dropped 284 log messages in last 1 seconds (most recently, 0 seconds ago) due to excessive rate
2020-06-02T16:01:11Z|00753|poll_loop|INFO|wakeup due to [POLLOUT] on fd 57 (10.0.221.58:9642<->10.0.218.162:45516) at lib/stream-ssl.c:793 (97% CPU usage)
2020-06-02T16:01:13Z|00754|raft|INFO|server db04 is leader for term 775
2020-06-02T16:01:41Z|00755|poll_loop|INFO|Dropped 106 log messages in last 30 seconds (most recently, 28 seconds ago) due to excessive rate
2020-06-02T16:01:41Z|00756|poll_loop|INFO|wakeup due to [POLLOUT] on fd 101 (10.0.221.58:9642<->10.0.203.111:33806) at lib/stream-ssl.c:793 (65% CPU usage)
2020-06-02T16:01:41Z|00757|poll_loop|INFO|wakeup due to [POLLOUT] on fd 77 (10.0.221.58:9642<->10.0.254.25:57982) at lib/stream-ssl.c:793 (65% CPU usage)
2020-06-02T16:01:41Z|00758|poll_loop|INFO|wakeup due to [POLLOUT] on fd 70 (10.0.221.58:9642<->10.0.187.141:55272) at lib/stream-ssl.c:793 (65% CPU usage)
2020-06-02T16:01:41Z|00759|poll_loop|INFO|wakeup due to [POLLOUT] on fd 59 (10.0.221.58:9642<->10.0.243.54:47020) at lib/stream-ssl.c:793 (65% CPU usage)
2020-06-02T16:01:41Z|00760|poll_loop|INFO|Dropped 55 log messages in last 0 seconds (most recently, 0 seconds ago) due to excessive rate
2020-06-02T16:01:41Z|00761|poll_loop|INFO|wakeup due to [POLLOUT] on fd 95 (10.0.221.58:9642<->10.0.240.137:58660) at lib/stream-ssl.c:793 (65% CPU usage)
2020-06-02T16:02:07Z|00762|poll_loop|INFO|Dropped 214 log messages in last 26 seconds (most recently, 24 seconds ago) due to excessive rate
2020-06-02T16:02:07Z|00763|poll_loop|INFO|wakeup due to [POLLOUT] on fd 37 (10.0.221.58:9642<->10.0.186.35:36894) at lib/stream-ssl.c:793 (63% CPU usage)
2020-06-02T16:02:07Z|00764|poll_loop|INFO|wakeup due to [POLLOUT] on fd 46 (10.0.221.58:9642<->10.0.243.175:52058) at lib/stream-ssl.c:793 (63% CPU usage)
2020-06-02T16:02:07Z|00765|poll_loop|INFO|wakeup due to [POLLOUT] on fd 94 (10.0.221.58:9642<->10.0.243.105:60164) at lib/stream-ssl.c:793 (63% CPU usage)
2020-06-02T16:02:07Z|00766|poll_loop|INFO|wakeup due to [POLLOUT] on fd 51 (10.0.221.58:9642<->10.0.181.159:33340) at lib/stream-ssl.c:793 (63% CPU usage)
2020-06-02T16:02:11Z|00767|poll_loop|INFO|Dropped 317 log messages in last 4 seconds (most recently, 0 seconds ago) due to excessive rate
2020-06-02T16:02:11Z|00768|poll_loop|INFO|wakeup due to [POLLIN] on fd 26 (10.0.221.58:9644<->10.0.173.114:59758) at lib/stream-ssl.c:827 (55% CPU usage)
2020-06-02T16:02:17Z|00769|reconnect|ERR|ssl:10.0.135.159:9644: no response to inactivity probe after 5.03 seconds, disconnecting
2020-06-02T16:02:17Z|00770|reconnect|INFO|ssl:10.0.135.159:9644: connection dropped
2020-06-02T16:02:17Z|00771|reconnect|ERR|ssl:10.0.135.159:50000: no response to inactivity probe after 5.03 seconds, disconnecting
2020-06-02T16:02:17Z|00772|raft|INFO|term 776: 1500 ms timeout expired, starting election
2020-06-02T16:02:17Z|00773|raft|INFO|term 776: elected leader by 2+ of 3 servers
2020-06-02T16:02:17Z|00774|poll_loop|INFO|Dropped 458 log messages in last 6 seconds (most recently, 0 seconds ago) due to excessive rate
2020-06-02T16:02:17Z|00775|poll_loop|INFO|wakeup due to 257-ms timeout at ovsdb/ovsdb-server.c:279 (60% CPU usage)
2020-06-02T16:02:18Z|00776|reconnect|INFO|ssl:10.0.135.159:9644: connecting...
2020-06-02T16:02:18Z|00777|reconnect|INFO|ssl:10.0.135.159:9644: connected
2020-06-02T16:02:18Z|00778|raft|INFO|ssl:10.0.135.159:51670: learned server ID 77d9
2020-06-02T16:02:18Z|00779|raft|INFO|ssl:10.0.135.159:51670: learned remote address ssl:10.0.135.159:9644
2020-06-02T16:02:18Z|00780|raft|WARN|Dropped 1 log messages in last 66 seconds (most recently, 66 seconds ago) due to excessive rate
2020-06-02T16:02:18Z|00781|raft|WARN|ignoring vote request received as leader
2020-06-02T16:02:18Z|00782|raft|INFO|rejected append_reply (not leader)
2020-06-02T16:02:19Z|00783|raft|WARN|ignoring vote request received after only 910 ms (minimum election time is 1000 ms)
2020-06-02T16:02:19Z|00784|raft|INFO|server db04 is leader for term 777
2020-06-02T16:02:20Z|00785|raft|INFO|term 778: elected leader by 2+ of 3 servers
2020-06-02T16:02:27Z|00786|raft|WARN|ignoring vote request received as leader
2020-06-02T16:02:27Z|00787|raft|INFO|server 77d9 is leader for term 779
2020-06-02T16:02:27Z|00788|timeval|WARN|Unreasonably long 1849ms poll interval (1699ms user, 137ms system)
2020-06-02T16:02:27Z|00789|timeval|WARN|faults: 36343 minor, 0 major
2020-06-02T16:02:27Z|00790|timeval|WARN|disk: 0 reads, 16 writes
2020-06-02T16:02:27Z|00791|timeval|WARN|context switches: 1 voluntary, 3 involuntary
2020-06-02T16:02:27Z|00792|coverage|INFO|Skipping details of duplicate event coverage for hash=1e253855
2020-06-02T16:02:27Z|00793|poll_loop|INFO|Dropped 79 log messages in last 10 seconds (most recently, 8 seconds ago) due to excessive rate
2020-06-02T16:02:27Z|00794|poll_loop|INFO|wakeup due to [POLLOUT] on fd 114 (10.0.221.58:9642<->10.0.167.40:43096) at lib/stream-ssl.c:793 (55% CPU usage)
2020-06-02T16:02:29Z|00795|poll_loop|INFO|Dropped 174 log messages in last 2 seconds (most recently, 0 seconds ago) due to excessive rate
2020-06-02T16:02:29Z|00796|poll_loop|INFO|wakeup due to [POLLOUT] on fd 115 (10.0.221.58:9642<->10.0.166.149:46952) at lib/stream-ssl.c:793 (55% CPU usage)
2020-06-02T16:02:34Z|00797|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:02:34Z|00798|jsonrpc|WARN|ssl:10.0.185.67:37282: receive error: Protocol error
2020-06-02T16:02:34Z|00799|reconnect|WARN|ssl:10.0.185.67:37282: connection dropped (Protocol error)
2020-06-02T16:02:34Z|00800|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:02:34Z|00801|jsonrpc|WARN|ssl:10.0.225.111:45256: receive error: Protocol error
2020-06-02T16:02:34Z|00802|reconnect|WARN|ssl:10.0.225.111:45256: connection dropped (Protocol error)
2020-06-02T16:02:34Z|00803|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:02:34Z|00804|jsonrpc|WARN|ssl:10.0.166.236:49600: receive error: Protocol error
2020-06-02T16:02:34Z|00805|reconnect|WARN|ssl:10.0.166.236:49600: connection dropped (Protocol error)
2020-06-02T16:02:34Z|00806|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:02:34Z|00807|jsonrpc|WARN|ssl:10.0.209.243:33964: receive error: Protocol error
2020-06-02T16:02:34Z|00808|reconnect|WARN|ssl:10.0.209.243:33964: connection dropped (Protocol error)
2020-06-02T16:02:34Z|00809|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:02:34Z|00810|jsonrpc|WARN|ssl:10.0.203.69:53370: receive error: Protocol error
2020-06-02T16:02:34Z|00811|reconnect|WARN|ssl:10.0.203.69:53370: connection dropped (Protocol error)
2020-06-02T16:02:34Z|00812|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:02:34Z|00813|jsonrpc|WARN|ssl:10.0.173.114:38374: receive error: Protocol error
2020-06-02T16:02:34Z|00814|reconnect|WARN|ssl:10.0.173.114:38374: connection dropped (Protocol error)
2020-06-02T16:02:34Z|00815|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:02:34Z|00816|reconnect|WARN|ssl:10.0.251.213:40366: connection dropped (Protocol error)
2020-06-02T16:02:34Z|00817|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:02:34Z|00818|reconnect|WARN|ssl:10.0.243.129:48220: connection dropped (Protocol error)
2020-06-02T16:02:39Z|00819|timeval|WARN|Unreasonably long 2529ms poll interval (2365ms user, 145ms system)
2020-06-02T16:02:39Z|00820|timeval|WARN|faults: 19164 minor, 0 major
2020-06-02T16:02:39Z|00821|timeval|WARN|context switches: 0 voluntary, 3 involuntary
2020-06-02T16:02:39Z|00822|poll_loop|INFO|Dropped 130 log messages in last 10 seconds (most recently, 7 seconds ago) due to excessive rate
2020-06-02T16:02:39Z|00823|poll_loop|INFO|wakeup due to [POLLOUT] on fd 101 (10.0.221.58:9642<->10.0.203.111:33806) at lib/stream-ssl.c:793 (91% CPU usage)
2020-06-02T16:02:41Z|00824|poll_loop|INFO|Dropped 191 log messages in last 2 seconds (most recently, 1 seconds ago) due to excessive rate
2020-06-02T16:02:41Z|00825|poll_loop|INFO|wakeup due to [POLLIN] on fd 25 (10.0.221.58:9644<->10.0.135.159:51670) at lib/stream-ssl.c:827 (91% CPU usage)
2020-06-02T16:08:19Z|00826|raft|WARN|ignoring vote request received after only 0 ms (minimum election time is 1000 ms)
2020-06-02T16:08:19Z|00827|timeval|WARN|Unreasonably long 4366ms poll interval (4056ms user, 280ms system)
2020-06-02T16:08:19Z|00828|timeval|WARN|faults: 152796 minor, 0 major
2020-06-02T16:08:19Z|00829|timeval|WARN|disk: 0 reads, 8 writes
2020-06-02T16:08:19Z|00830|timeval|WARN|context switches: 0 voluntary, 6 involuntary
2020-06-02T16:08:19Z|00831|coverage|INFO|Dropped 1 log messages in last 339 seconds (most recently, 339 seconds ago) due to excessive rate
2020-06-02T16:08:19Z|00832|coverage|INFO|Skipping details of duplicate event coverage for hash=1e253855
2020-06-02T16:08:19Z|00833|poll_loop|INFO|Dropped 16 log messages in last 337 seconds (most recently, 336 seconds ago) due to excessive rate
2020-06-02T16:08:19Z|00834|poll_loop|INFO|wakeup due to [POLLIN] on fd 59 (10.0.221.58:9642<->10.0.243.54:47020) at lib/stream-ssl.c:827 (85% CPU usage)
2020-06-02T16:08:19Z|00835|poll_loop|INFO|wakeup due to [POLLIN] on fd 13 (0.0.0.0:9642<->) at lib/stream-ssl.c:963 (85% CPU usage)
2020-06-02T16:08:19Z|00836|poll_loop|INFO|wakeup due to [POLLIN] on fd 89 (10.0.221.58:9642<->10.0.208.133:34190) at lib/stream-ssl.c:827 (85% CPU usage)
2020-06-02T16:08:19Z|00837|poll_loop|INFO|wakeup due to [POLLIN] on fd 80 (10.0.221.58:9642<->10.0.200.93:39116) at lib/stream-ssl.c:827 (85% CPU usage)
2020-06-02T16:08:19Z|00838|poll_loop|INFO|wakeup due to [POLLIN] on fd 76 (10.0.221.58:9642<->10.0.246.153:53298) at lib/stream-ssl.c:827 (85% CPU usage)
2020-06-02T16:08:19Z|00839|poll_loop|INFO|wakeup due to [POLLIN] on fd 74 (10.0.221.58:9642<->10.0.233.45:34372) at lib/stream-ssl.c:827 (85% CPU usage)
2020-06-02T16:08:19Z|00840|poll_loop|INFO|wakeup due to [POLLIN] on fd 81 (10.0.221.58:9642<->10.0.253.58:37330) at lib/stream-ssl.c:827 (85% CPU usage)
2020-06-02T16:08:19Z|00841|poll_loop|INFO|wakeup due to [POLLIN] on fd 31 (10.0.221.58:9642<->10.0.230.162:43582) at lib/stream-ssl.c:827 (85% CPU usage)
2020-06-02T16:08:19Z|00842|poll_loop|INFO|wakeup due to [POLLIN] on fd 77 (10.0.221.58:9642<->10.0.254.25:57982) at lib/stream-ssl.c:827 (85% CPU usage)
2020-06-02T16:08:19Z|00843|poll_loop|INFO|wakeup due to [POLLOUT] on fd 147 (10.0.221.58:9642<->10.0.198.203:38492) at lib/stream-ssl.c:793 (85% CPU usage)
2020-06-02T16:08:20Z|00844|raft|INFO|Dropped 1 log messages in last 361 seconds (most recently, 361 seconds ago) due to excessive rate
2020-06-02T16:08:20Z|00845|raft|INFO|term 780: 1111 ms timeout expired, starting election
2020-06-02T16:08:22Z|00846|raft|INFO|server 77d9 is leader for term 781
2020-06-02T16:08:22Z|00847|memory|INFO|peak resident set size grew 57% in last 517.2 seconds, from 1421132 kB to 2227188 kB
2020-06-02T16:08:22Z|00848|memory|INFO|cells:3031563 monitors:2 sessions:122
2020-06-02T16:08:23Z|00849|raft|WARN|ignoring vote request received after only 52 ms (minimum election time is 1000 ms)
2020-06-02T16:08:31Z|00850|raft|INFO|term 782: 1198 ms timeout expired, starting election
2020-06-02T16:08:31Z|00851|raft|INFO|term 782: elected leader by 2+ of 3 servers
2020-06-02T16:08:35Z|00852|raft|WARN|ignoring vote request received as leader
2020-06-02T16:08:35Z|00853|raft|WARN|ignoring vote request received as leader
2020-06-02T16:08:35Z|00854|timeval|WARN|Unreasonably long 4266ms poll interval (4010ms user, 226ms system)
2020-06-02T16:08:35Z|00855|timeval|WARN|faults: 117668 minor, 0 major
2020-06-02T16:08:35Z|00856|timeval|WARN|context switches: 0 voluntary, 6 involuntary
2020-06-02T16:08:35Z|00857|coverage|INFO|Skipping details of duplicate event coverage for hash=1e253855
2020-06-02T16:08:35Z|00858|poll_loop|INFO|Dropped 381 log messages in last 16 seconds (most recently, 13 seconds ago) due to excessive rate
2020-06-02T16:08:35Z|00859|poll_loop|INFO|wakeup due to [POLLIN] on fd 13 (0.0.0.0:9642<->) at lib/stream-ssl.c:963 (63% CPU usage)
2020-06-02T16:08:35Z|00860|poll_loop|INFO|wakeup due to [POLLIN] on fd 14 (/var/run/ovn/ovnsb_db.ctl<->) at lib/stream-fd.c:274 (63% CPU usage)
2020-06-02T16:08:35Z|00861|jsonrpc|WARN|Dropped 2 log messages in last 361 seconds (most recently, 361 seconds ago) due to excessive rate
2020-06-02T16:08:35Z|00862|jsonrpc|WARN|unix#231: send error: Broken pipe
2020-06-02T16:08:35Z|00863|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:08:35Z|00864|jsonrpc|WARN|ssl:10.0.143.177:36166: receive error: Protocol error
2020-06-02T16:08:35Z|00865|raft|INFO|rejected append_reply (not leader)
2020-06-02T16:08:35Z|00866|raft|INFO|rejecting term 781 < current term 784 received in vote_reply message from server 77d9
2020-06-02T16:08:35Z|00867|raft|INFO|rejecting term 782 < current term 784 received in append_reply message from server 77d9
2020-06-02T16:08:35Z|00868|raft|INFO|rejecting term 782 < current term 784 received in append_reply message from server 77d9
2020-06-02T16:08:35Z|00869|raft|INFO|rejecting term 782 < current term 784 received in append_reply message from server 77d9
2020-06-02T16:08:35Z|00870|reconnect|WARN|ssl:10.0.143.177:36166: connection dropped (Protocol error)
2020-06-02T16:08:35Z|00871|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:08:35Z|00872|jsonrpc|WARN|ssl:10.0.142.155:48588: receive error: Protocol error
2020-06-02T16:08:35Z|00873|reconnect|WARN|ssl:10.0.142.155:48588: connection dropped (Protocol error)
2020-06-02T16:08:35Z|00874|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:08:35Z|00875|jsonrpc|WARN|ssl:10.0.146.2:33288: receive error: Protocol error
2020-06-02T16:08:35Z|00876|reconnect|WARN|ssl:10.0.146.2:33288: connection dropped (Protocol error)
2020-06-02T16:08:35Z|00877|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:08:35Z|00878|jsonrpc|WARN|ssl:10.0.173.114:51178: receive error: Protocol error
2020-06-02T16:08:35Z|00879|reconnect|WARN|ssl:10.0.173.114:51178: connection dropped (Protocol error)
2020-06-02T16:08:35Z|00880|raft|WARN|ignoring vote request received after only 140 ms (minimum election time is 1000 ms)
2020-06-02T16:08:36Z|00881|raft|INFO|term 785: 1406 ms timeout expired, starting election
2020-06-02T16:08:37Z|00882|poll_loop|INFO|Dropped 358 log messages in last 2 seconds (most recently, 0 seconds ago) due to excessive rate
2020-06-02T16:08:37Z|00883|poll_loop|INFO|wakeup due to 364-ms timeout at lib/reconnect.c:643 (63% CPU usage)
2020-06-02T16:08:38Z|00884|raft|INFO|term 787: 1145 ms timeout expired, starting election
2020-06-02T16:08:38Z|00885|raft|INFO|term 787: elected leader by 2+ of 3 servers
2020-06-02T16:08:38Z|00886|raft|INFO|rejecting term 785 < current term 787 received in append_reply message from server 77d9
2020-06-02T16:08:38Z|00887|raft|INFO|rejecting term 785 < current term 787 received in vote_reply message from server 77d9
2020-06-02T16:08:38Z|00888|raft|INFO|rejecting term 785 < current term 787 received in vote_reply message from server 77d9
2020-06-02T16:08:38Z|00889|raft|INFO|current entry eid 9b7d5e5b-e105-4baf-b590-ee61e145e8c7 does not match prerequisite 2171c08f-e056-41e0-80cc-5cbd4739460c in execute_command_request
2020-06-02T16:08:38Z|00890|raft|INFO|current entry eid 9b7d5e5b-e105-4baf-b590-ee61e145e8c7 does not match prerequisite 2171c08f-e056-41e0-80cc-5cbd4739460c in execute_command_request
2020-06-02T16:08:38Z|00891|raft|INFO|current entry eid 1cb84b0f-8205-48e2-bc5b-1f5f839797a1 does not match prerequisite 9b7d5e5b-e105-4baf-b590-ee61e145e8c7 in execute_command_request
2020-06-02T16:08:38Z|00892|raft|INFO|current entry eid 1cb84b0f-8205-48e2-bc5b-1f5f839797a1 does not match prerequisite 9b7d5e5b-e105-4baf-b590-ee61e145e8c7 in execute_command_request
2020-06-02T16:08:49Z|00893|raft|WARN|ignoring vote request received as leader
2020-06-02T16:08:49Z|00894|raft|WARN|ignoring vote request received as leader
2020-06-02T16:08:49Z|00895|raft|WARN|ignoring vote request received as leader
2020-06-02T16:08:49Z|00896|timeval|WARN|Unreasonably long 4383ms poll interval (4069ms user, 255ms system)
2020-06-02T16:08:49Z|00897|timeval|WARN|faults: 113802 minor, 0 major
2020-06-02T16:08:49Z|00898|timeval|WARN|context switches: 0 voluntary, 125 involuntary
2020-06-02T16:08:49Z|00899|coverage|INFO|Skipping details of duplicate event coverage for hash=1e253855
2020-06-02T16:08:49Z|00900|poll_loop|INFO|Dropped 32 log messages in last 12 seconds (most recently, 12 seconds ago) due to excessive rate
2020-06-02T16:08:49Z|00901|poll_loop|INFO|wakeup due to [POLLIN][POLLERR][POLLHUP] on fd 133 (10.0.221.58:9642<->) at lib/stream-ssl.c:827 (82% CPU usage)
2020-06-02T16:08:49Z|00902|poll_loop|INFO|wakeup due to [POLLIN][POLLERR][POLLHUP] on fd 114 (10.0.221.58:9642<->) at lib/stream-ssl.c:827 (82% CPU usage)
2020-06-02T16:08:49Z|00903|stream_ssl|WARN|SSL_write: system error (Connection reset by peer)
2020-06-02T16:08:49Z|00904|jsonrpc|WARN|ssl:10.0.163.90:47478: send error: Connection reset by peer
2020-06-02T16:08:49Z|00905|stream_ssl|WARN|SSL_write: system error (Connection reset by peer)
2020-06-02T16:08:49Z|00906|stream_ssl|WARN|SSL_write: system error (Connection reset by peer)
2020-06-02T16:08:49Z|00907|stream_ssl|WARN|SSL_write: system error (Connection reset by peer)
2020-06-02T16:08:49Z|00908|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:08:49Z|00909|raft|INFO|rejected append_reply (not leader)
2020-06-02T16:08:49Z|00910|raft|INFO|rejected append_reply (not leader)
2020-06-02T16:08:49Z|00911|reconnect|WARN|ssl:10.0.163.90:47478: connection dropped (Connection reset by peer)
2020-06-02T16:08:49Z|00912|reconnect|WARN|ssl:10.0.185.138:32934: connection dropped (Connection reset by peer)
2020-06-02T16:08:49Z|00913|reconnect|WARN|ssl:10.0.161.71:58588: connection dropped (Connection reset by peer)
2020-06-02T16:08:49Z|00914|reconnect|WARN|ssl:10.0.240.137:58660: connection dropped (Connection reset by peer)
2020-06-02T16:08:49Z|00915|reconnect|WARN|ssl:10.0.158.112:55964: connection dropped (Protocol error)
2020-06-02T16:08:49Z|00916|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:08:49Z|00917|reconnect|WARN|ssl:10.0.173.114:52736: connection dropped (Protocol error)
2020-06-02T16:08:54Z|00918|raft|INFO|term 791: 4281 ms timeout expired, starting election
2020-06-02T16:08:54Z|00919|timeval|WARN|Unreasonably long 4266ms poll interval (4022ms user, 213ms system)
2020-06-02T16:08:54Z|00920|timeval|WARN|faults: 113021 minor, 0 major
2020-06-02T16:08:54Z|00921|timeval|WARN|disk: 0 reads, 8 writes
2020-06-02T16:08:54Z|00922|timeval|WARN|context switches: 0 voluntary, 134 involuntary
2020-06-02T16:08:54Z|00923|stream_ssl|WARN|SSL_write: system error (Connection reset by peer)
2020-06-02T16:08:54Z|00924|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:08:54Z|00925|raft|INFO|server db04 is leader for term 791
2020-06-02T16:08:54Z|00926|raft|INFO|90d3 truncating 1 entries from end of log
2020-06-02T16:08:54Z|00927|raft|INFO|rejecting term 788 < current term 791 received in append_reply message from server 77d9
2020-06-02T16:08:54Z|00928|raft|INFO|rejecting term 788 < current term 791 received in append_reply message from server 77d9
2020-06-02T16:08:54Z|00929|reconnect|WARN|ssl:10.0.165.227:51760: connection dropped (Connection reset by peer)
2020-06-02T16:08:54Z|00930|reconnect|WARN|ssl:10.0.191.153:43882: connection dropped (Protocol error)
2020-06-02T16:08:54Z|00931|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:08:54Z|00932|reconnect|WARN|ssl:10.0.147.59:57660: connection dropped (Protocol error)
2020-06-02T16:08:54Z|00933|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:08:54Z|00934|reconnect|WARN|ssl:10.0.134.200:50062: connection dropped (Protocol error)
2020-06-02T16:08:54Z|00935|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:08:54Z|00936|reconnect|WARN|ssl:10.0.221.58:35490: connection dropped (Protocol error)
2020-06-02T16:08:58Z|00937|timeval|WARN|Unreasonably long 4153ms poll interval (4056ms user, 64ms system)
2020-06-02T16:08:58Z|00938|timeval|WARN|faults: 23787 minor, 0 major
2020-06-02T16:08:58Z|00939|timeval|WARN|context switches: 0 voluntary, 174 involuntary
2020-06-02T16:08:58Z|00940|poll_loop|INFO|Dropped 117 log messages in last 8 seconds (most recently, 4 seconds ago) due to excessive rate
2020-06-02T16:08:58Z|00941|poll_loop|INFO|wakeup due to [POLLOUT] on fd 27 (10.0.221.58:9642<->10.0.135.212:35902) at lib/stream-ssl.c:793 (100% CPU usage)
2020-06-02T16:08:58Z|00942|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:08:58Z|00943|reconnect|WARN|ssl:10.0.135.159:46194: connection dropped (Protocol error)
2020-06-02T16:08:58Z|00944|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:08:58Z|00945|reconnect|WARN|ssl:10.0.128.209:39794: connection dropped (Protocol error)
2020-06-02T16:08:58Z|00946|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:08:58Z|00947|reconnect|WARN|ssl:10.0.137.94:53830: connection dropped (Protocol error)
2020-06-02T16:09:04Z|00948|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:09:04Z|00949|jsonrpc|WARN|Dropped 14 log messages in last 15 seconds (most recently, 7 seconds ago) due to excessive rate
2020-06-02T16:09:04Z|00950|jsonrpc|WARN|ssl:10.0.135.159:57032: receive error: Protocol error
2020-06-02T16:09:04Z|00951|timeval|WARN|Unreasonably long 4149ms poll interval (4098ms user, 21ms system)
2020-06-02T16:09:04Z|00952|timeval|WARN|faults: 49 minor, 0 major
2020-06-02T16:09:04Z|00953|timeval|WARN|context switches: 0 voluntary, 9 involuntary
2020-06-02T16:09:04Z|00954|poll_loop|INFO|Dropped 17085 log messages in last 7 seconds (most recently, 5 seconds ago) due to excessive rate
2020-06-02T16:09:04Z|00955|poll_loop|INFO|wakeup due to [POLLIN] on fd 27 (10.0.221.58:9642<->10.0.135.212:35902) at lib/stream-ssl.c:827 (87% CPU usage)
2020-06-02T16:09:04Z|00956|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:09:04Z|00957|reconnect|WARN|ssl:10.0.135.159:57032: connection dropped (Protocol error)
2020-06-02T16:09:04Z|00958|reconnect|WARN|ssl:10.0.137.94:53850: connection dropped (Protocol error)
2020-06-02T16:09:10Z|00959|raft|INFO|ssl:10.0.135.159:57074: learned server ID 77d9
2020-06-02T16:09:10Z|00960|raft|INFO|ssl:10.0.135.159:57074: learned remote address ssl:10.0.135.159:9644
2020-06-02T16:09:10Z|00961|raft|WARN|ignoring vote request received after only 1 ms (minimum election time is 1000 ms)
2020-06-02T16:09:10Z|00962|timeval|WARN|Unreasonably long 4134ms poll interval (4086ms user, 21ms system)
2020-06-02T16:09:10Z|00963|timeval|WARN|faults: 50 minor, 0 major
2020-06-02T16:09:10Z|00964|timeval|WARN|disk: 0 reads, 8 writes
2020-06-02T16:09:10Z|00965|timeval|WARN|context switches: 0 voluntary, 15 involuntary
2020-06-02T16:09:10Z|00966|poll_loop|INFO|Dropped 9474 log messages in last 6 seconds (most recently, 4 seconds ago) due to excessive rate
2020-06-02T16:09:10Z|00967|poll_loop|INFO|wakeup due to [POLLIN] on fd 13 (0.0.0.0:9642<->) at lib/stream-ssl.c:963 (100% CPU usage)
2020-06-02T16:09:10Z|00968|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:09:10Z|00969|reconnect|WARN|ssl:10.0.135.83:52450: connection dropped (Protocol error)
2020-06-02T16:09:10Z|00970|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:09:10Z|00971|reconnect|WARN|ssl:10.0.139.136:41578: connection dropped (Protocol error)
2020-06-02T16:09:10Z|00972|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:09:10Z|00973|reconnect|WARN|ssl:10.0.131.177:38106: connection dropped (Protocol error)
2020-06-02T16:09:10Z|00974|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:09:10Z|00975|reconnect|WARN|ssl:10.0.128.101:56180: connection dropped (Protocol error)
2020-06-02T16:09:10Z|00976|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:09:10Z|00977|reconnect|WARN|ssl:10.0.156.245:33914: connection dropped (Protocol error)
2020-06-02T16:09:10Z|00978|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:09:10Z|00979|reconnect|WARN|ssl:10.0.130.210:39660: connection dropped (Protocol error)
2020-06-02T16:09:10Z|00980|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:09:10Z|00981|reconnect|WARN|ssl:10.0.131.198:41666: connection dropped (Protocol error)
2020-06-02T16:09:10Z|00982|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:09:10Z|00983|reconnect|WARN|ssl:10.0.142.93:55354: connection dropped (Protocol error)
2020-06-02T16:09:14Z|00984|raft|WARN|ignoring vote request received after only 0 ms (minimum election time is 1000 ms)
2020-06-02T16:09:14Z|00985|timeval|WARN|Unreasonably long 4084ms poll interval (4009ms user, 47ms system)
2020-06-02T16:09:14Z|00986|timeval|WARN|faults: 16139 minor, 0 major
2020-06-02T16:09:14Z|00987|timeval|WARN|disk: 0 reads, 8 writes
2020-06-02T16:09:14Z|00988|timeval|WARN|context switches: 0 voluntary, 5 involuntary
2020-06-02T16:09:14Z|00989|poll_loop|INFO|Dropped 17 log messages in last 4 seconds (most recently, 4 seconds ago) due to excessive rate
2020-06-02T16:09:14Z|00990|poll_loop|INFO|wakeup due to [POLLIN] on fd 13 (0.0.0.0:9642<->) at lib/stream-ssl.c:963 (99% CPU usage)
2020-06-02T16:09:14Z|00991|jsonrpc|WARN|Dropped 10 log messages in last 10 seconds (most recently, 4 seconds ago) due to excessive rate
2020-06-02T16:09:14Z|00992|jsonrpc|WARN|unix#235: send error: Broken pipe
2020-06-02T16:09:14Z|00993|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:09:14Z|00994|reconnect|WARN|ssl:10.0.173.114:54804: connection dropped (Protocol error)
2020-06-02T16:09:14Z|00995|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:09:14Z|00996|reconnect|WARN|ssl:10.0.154.176:57348: connection dropped (Protocol error)
2020-06-02T16:09:14Z|00997|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:09:14Z|00998|reconnect|WARN|ssl:10.0.182.146:53186: connection dropped (Protocol error)
2020-06-02T16:09:14Z|00999|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:09:14Z|01000|reconnect|WARN|ssl:10.0.138.166:46136: connection dropped (Protocol error)
2020-06-02T16:09:14Z|01001|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:09:14Z|01002|reconnect|WARN|ssl:10.0.135.159:46458: connection dropped (Protocol error)
2020-06-02T16:09:14Z|01003|raft|INFO|rejecting term 791 < current term 796 received in append_request message from server db04
2020-06-02T16:09:15Z|01004|raft|INFO|rejecting term 791 < current term 796 received in append_request message from server db04
2020-06-02T16:09:15Z|01005|raft|INFO|rejecting term 791 < current term 796 received in append_request message from server db04
2020-06-02T16:09:15Z|01006|raft|INFO|rejecting term 793 < current term 796 received in vote_reply message from server db04
2020-06-02T16:09:16Z|01007|raft|INFO|rejecting term 796 < current term 797 received in vote_reply message from server db04
2020-06-02T16:09:16Z|01008|raft|INFO|server 77d9 is leader for term 799
2020-06-02T16:09:16Z|01009|raft|INFO|rejecting append_request because previous entry 791,85370 not in local log (mismatch past end of log)
2020-06-02T16:09:16Z|01010|raft|INFO|rejecting append_request because previous entry 791,85370 not in local log (mismatch past end of log)
2020-06-02T16:09:21Z|01011|timeval|WARN|Unreasonably long 4058ms poll interval (3928ms user, 80ms system)
2020-06-02T16:09:21Z|01012|timeval|WARN|faults: 21681 minor, 0 major
2020-06-02T16:09:21Z|01013|timeval|WARN|context switches: 0 voluntary, 7 involuntary
2020-06-02T16:09:21Z|01014|coverage|INFO|Dropped 5 log messages in last 27 seconds (most recently, 6 seconds ago) due to excessive rate
2020-06-02T16:09:21Z|01015|coverage|INFO|Skipping details of duplicate event coverage for hash=1e253855
2020-06-02T16:09:21Z|01016|poll_loop|INFO|Dropped 4495 log messages in last 6 seconds (most recently, 4 seconds ago) due to excessive rate
2020-06-02T16:09:21Z|01017|poll_loop|INFO|wakeup due to [POLLIN] on fd 13 (0.0.0.0:9642<->) at lib/stream-ssl.c:963 (79% CPU usage)
2020-06-02T16:09:21Z|01018|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:09:21Z|01019|reconnect|WARN|ssl:10.0.139.121:43496: connection dropped (Protocol error)
2020-06-02T16:09:21Z|01020|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:09:25Z|01021|reconnect|WARN|ssl:10.0.138.181:56472: connection dropped (Protocol error)
2020-06-02T16:09:25Z|01022|raft|WARN|Dropped 3 log messages in last 11 seconds (most recently, 5 seconds ago) due to excessive rate
2020-06-02T16:09:25Z|01023|raft|WARN|ignoring vote request received after only 0 ms (minimum election time is 1000 ms)
2020-06-02T16:09:25Z|01024|timeval|WARN|Unreasonably long 4172ms poll interval (3981ms user, 140ms system)
2020-06-02T16:09:25Z|01025|timeval|WARN|faults: 46294 minor, 0 major
2020-06-02T16:09:25Z|01026|timeval|WARN|disk: 0 reads, 8 writes
2020-06-02T16:09:25Z|01027|timeval|WARN|context switches: 0 voluntary, 8 involuntary
2020-06-02T16:09:25Z|01028|poll_loop|INFO|Dropped 16 log messages in last 5 seconds (most recently, 5 seconds ago) due to excessive rate
2020-06-02T16:09:25Z|01029|poll_loop|INFO|wakeup due to [POLLIN] on fd 37 (10.0.221.58:9642<->10.0.208.175:48782) at lib/stream-ssl.c:827 (98% CPU usage)
2020-06-02T16:09:25Z|01030|jsonrpc|WARN|Dropped 7 log messages in last 11 seconds (most recently, 5 seconds ago) due to excessive rate
2020-06-02T16:09:25Z|01031|jsonrpc|WARN|unix#236: send error: Broken pipe
2020-06-02T16:09:25Z|01032|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:09:25Z|01033|reconnect|WARN|ssl:10.0.129.0:55140: connection dropped (Protocol error)
2020-06-02T16:09:25Z|01034|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:09:25Z|01035|reconnect|WARN|ssl:10.0.135.212:36008: connection dropped (Protocol error)
2020-06-02T16:09:30Z|01036|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:09:30Z|01037|timeval|WARN|Unreasonably long 4199ms poll interval (4008ms user, 134ms system)
2020-06-02T16:09:30Z|01038|timeval|WARN|faults: 46624 minor, 0 major
2020-06-02T16:09:30Z|01039|timeval|WARN|disk: 0 reads, 8 writes
2020-06-02T16:09:30Z|01040|timeval|WARN|context switches: 0 voluntary, 9 involuntary
2020-06-02T16:09:30Z|01041|reconnect|WARN|ssl:10.0.166.236:51408: connection dropped (Protocol error)
2020-06-02T16:09:30Z|01042|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:09:30Z|01043|reconnect|WARN|ssl:10.0.130.210:39690: connection dropped (Protocol error)
2020-06-02T16:09:30Z|01044|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:09:30Z|01045|reconnect|WARN|ssl:10.0.203.69:55202: connection dropped (Protocol error)
2020-06-02T16:09:30Z|01046|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:09:30Z|01047|reconnect|WARN|ssl:10.0.147.59:57760: connection dropped (Protocol error)
2020-06-02T16:09:30Z|01048|reconnect|WARN|ssl:10.0.139.96:59568: connection dropped (Protocol error)
2020-06-02T16:09:30Z|01049|reconnect|WARN|ssl:10.0.173.114:55880: connection dropped (Protocol error)
2020-06-02T16:09:30Z|01050|reconnect|WARN|ssl:10.0.191.153:44048: connection dropped (Protocol error)
2020-06-02T16:09:30Z|01051|reconnect|WARN|ssl:10.0.221.58:35840: connection dropped (Protocol error)
2020-06-02T16:09:30Z|01052|reconnect|WARN|ssl:10.0.135.159:46664: connection dropped (Protocol error)
2020-06-02T16:09:34Z|01053|timeval|WARN|Unreasonably long 4119ms poll interval (3972ms user, 88ms system)
2020-06-02T16:09:34Z|01054|timeval|WARN|faults: 26371 minor, 0 major
2020-06-02T16:09:34Z|01055|timeval|WARN|disk: 0 reads, 8 writes
2020-06-02T16:09:34Z|01056|timeval|WARN|context switches: 0 voluntary, 8 involuntary
2020-06-02T16:09:34Z|01057|poll_loop|INFO|Dropped 3760 log messages in last 9 seconds (most recently, 4 seconds ago) due to excessive rate
2020-06-02T16:09:34Z|01058|poll_loop|INFO|wakeup due to [POLLOUT] on fd 37 (10.0.221.58:9642<->10.0.154.176:57406) at lib/stream-ssl.c:793 (100% CPU usage)
2020-06-02T16:09:34Z|01059|stream_ssl|WARN|Dropped 5 log messages in last 5 seconds (most recently, 5 seconds ago) due to excessive rate
2020-06-02T16:09:34Z|01060|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:09:34Z|01061|reconnect|WARN|ssl:10.0.173.114:56236: connection dropped (Protocol error)
2020-06-02T16:09:34Z|01062|reconnect|WARN|ssl:10.0.242.24:51828: connection dropped (Protocol error)
2020-06-02T16:09:34Z|01063|reconnect|WARN|ssl:10.0.218.230:59068: connection dropped (Protocol error)
2020-06-02T16:09:34Z|01064|reconnect|WARN|ssl:10.0.135.159:46730: connection dropped (Protocol error)
2020-06-02T16:09:34Z|01065|reconnect|WARN|ssl:10.0.144.205:50800: connection dropped (Protocol error)
2020-06-02T16:09:35Z|01066|reconnect|WARN|ssl:10.0.135.159:57074: connection dropped (Protocol error)
2020-06-02T16:09:39Z|01067|raft|WARN|Dropped 5 log messages in last 13 seconds (most recently, 4 seconds ago) due to excessive rate
2020-06-02T16:09:39Z|01068|raft|WARN|ignoring vote request received after only 0 ms (minimum election time is 1000 ms)
2020-06-02T16:09:39Z|01069|raft|INFO|server db04 is leader for term 813
2020-06-02T16:09:39Z|01070|raft|INFO|rejecting append_request because previous entry 799,85372 not in local log (mismatch past end of log)
2020-06-02T16:09:39Z|01071|raft|INFO|rejecting append_request because previous entry 799,85372 not in local log (mismatch past end of log)
2020-06-02T16:09:39Z|01072|raft|INFO|rejecting append_request because previous entry 813,85373 not in local log (mismatch past end of log)
2020-06-02T16:09:39Z|01073|stream_ssl|WARN|Dropped 5 log messages in last 4 seconds (most recently, 4 seconds ago) due to excessive rate
2020-06-02T16:09:39Z|01074|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:09:39Z|01075|jsonrpc|WARN|Dropped 18 log messages in last 13 seconds (most recently, 4 seconds ago) due to excessive rate
2020-06-02T16:09:39Z|01076|jsonrpc|WARN|ssl:10.0.135.159:57568: receive error: Protocol error
2020-06-02T16:09:39Z|01077|timeval|WARN|Unreasonably long 4098ms poll interval (3977ms user, 69ms system)
2020-06-02T16:09:39Z|01078|timeval|WARN|faults: 8554 minor, 0 major
2020-06-02T16:09:39Z|01079|timeval|WARN|disk: 0 reads, 16 writes
2020-06-02T16:09:39Z|01080|timeval|WARN|context switches: 0 voluntary, 5 involuntary
2020-06-02T16:09:39Z|01081|poll_loop|INFO|Dropped 69 log messages in last 4 seconds (most recently, 4 seconds ago) due to excessive rate
2020-06-02T16:09:39Z|01082|poll_loop|INFO|wakeup due to [POLLOUT] on fd 37 (10.0.221.58:9642<->10.0.154.176:57406) at lib/stream-ssl.c:793 (100% CPU usage)
2020-06-02T16:09:39Z|01083|reconnect|WARN|ssl:10.0.135.159:57568: connection dropped (Protocol error)
2020-06-02T16:09:39Z|01084|raft|INFO|ssl:10.0.135.159:57616: learned server ID 77d9
2020-06-02T16:09:39Z|01085|raft|INFO|ssl:10.0.135.159:57616: learned remote address ssl:10.0.135.159:9644
2020-06-02T16:09:44Z|01086|raft|INFO|Dropped 2 log messages in last 38 seconds (most recently, 28 seconds ago) due to excessive rate
2020-06-02T16:09:44Z|01087|raft|INFO|term 814: 4842 ms timeout expired, starting election
2020-06-02T16:09:44Z|01088|timeval|WARN|Unreasonably long 4232ms poll interval (4032ms user, 99ms system)
2020-06-02T16:09:44Z|01089|timeval|WARN|faults: 49 minor, 0 major
2020-06-02T16:09:44Z|01090|timeval|WARN|disk: 0 reads, 8 writes
2020-06-02T16:09:44Z|01091|timeval|WARN|context switches: 0 voluntary, 78 involuntary
2020-06-02T16:09:44Z|01092|poll_loop|INFO|Dropped 173 log messages in last 5 seconds (most recently, 4 seconds ago) due to excessive rate
2020-06-02T16:09:44Z|01093|poll_loop|INFO|wakeup due to [POLLIN] on fd 37 (10.0.221.58:9642<->10.0.154.176:57406) at lib/stream-ssl.c:827 (90% CPU usage)
2020-06-02T16:09:44Z|01094|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:09:44Z|01095|raft|INFO|rejecting term 813 < current term 814 received in append_request message from server db04
2020-06-02T16:09:44Z|01096|raft|INFO|90d3 received "prerequisite check failed" reply from db04 for unknown command
2020-06-02T16:09:44Z|01097|raft|INFO|90d3 received "prerequisite check failed" reply from db04 for unknown command
2020-06-02T16:09:44Z|01098|raft|INFO|90d3 received "prerequisite check failed" reply from db04 for unknown command
2020-06-02T16:09:44Z|01099|reconnect|WARN|ssl:10.0.221.58:35904: connection dropped (Protocol error)
2020-06-02T16:09:44Z|01100|reconnect|WARN|ssl:10.0.173.114:57004: connection dropped (Protocol error)
2020-06-02T16:09:44Z|01101|reconnect|WARN|ssl:10.0.187.11:54104: connection dropped (Protocol error)
2020-06-02T16:09:44Z|01102|reconnect|WARN|ssl:10.0.203.111:35928: connection dropped (Protocol error)
2020-06-02T16:09:44Z|01103|reconnect|WARN|ssl:10.0.210.202:38808: connection dropped (Protocol error)
2020-06-02T16:09:44Z|01104|reconnect|WARN|ssl:10.0.251.235:34718: connection dropped (Protocol error)
2020-06-02T16:09:44Z|01105|raft|INFO|server 77d9 is leader for term 815
2020-06-02T16:09:44Z|01106|raft|INFO|rejecting append_request because previous entry 813,85373 not in local log (mismatch past end of log)
2020-06-02T16:09:44Z|01107|raft|INFO|rejecting append_request because previous entry 813,85373 not in local log (mismatch past end of log)
2020-06-02T16:09:48Z|01108|timeval|WARN|Unreasonably long 4077ms poll interval (3935ms user, 67ms system)
2020-06-02T16:09:48Z|01109|timeval|WARN|faults: 49 minor, 0 major
2020-06-02T16:09:48Z|01110|timeval|WARN|context switches: 0 voluntary, 9 involuntary
2020-06-02T16:09:48Z|01111|stream_ssl|WARN|Dropped 5 log messages in last 5 seconds (most recently, 5 seconds ago) due to excessive rate
2020-06-02T16:09:48Z|01112|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:09:48Z|01113|jsonrpc|WARN|Dropped 7 log messages in last 5 seconds (most recently, 5 seconds ago) due to excessive rate
2020-06-02T16:09:48Z|01114|jsonrpc|WARN|ssl:10.0.188.1:41370: receive error: Protocol error
2020-06-02T16:09:48Z|01115|reconnect|WARN|ssl:10.0.188.1:41370: connection dropped (Protocol error)
2020-06-02T16:09:48Z|01116|reconnect|WARN|ssl:10.0.152.22:46944: connection dropped (Protocol error)
2020-06-02T16:09:50Z|01117|poll_loop|INFO|Dropped 872 log messages in last 6 seconds (most recently, 1 seconds ago) due to excessive rate
2020-06-02T16:09:50Z|01118|poll_loop|INFO|wakeup due to [POLLIN] on fd 13 (0.0.0.0:9642<->) at lib/stream-ssl.c:963 (92% CPU usage)
2020-06-02T16:09:53Z|01119|timeval|WARN|Unreasonably long 3581ms poll interval (3546ms user, 0ms system)
2020-06-02T16:09:53Z|01120|timeval|WARN|disk: 0 reads, 8 writes
2020-06-02T16:09:53Z|01121|timeval|WARN|context switches: 0 voluntary, 7 involuntary
2020-06-02T16:09:53Z|01122|reconnect|WARN|ssl:10.0.144.109:36712: connection dropped (Input/output error)
2020-06-02T16:09:53Z|01123|reconnect|WARN|ssl:10.0.173.114:58018: connection dropped (Protocol error)
2020-06-02T16:09:53Z|01124|reconnect|WARN|ssl:10.0.221.58:36098: connection dropped (Protocol error)
2020-06-02T16:09:53Z|01125|reconnect|WARN|ssl:10.0.167.40:45040: connection dropped (Protocol error)
2020-06-02T16:09:53Z|01126|reconnect|WARN|ssl:10.0.237.106:47958: connection dropped (Protocol error)
2020-06-02T16:09:53Z|01127|reconnect|WARN|ssl:10.0.254.230:52076: connection dropped (Protocol error)
2020-06-02T16:09:53Z|01128|reconnect|WARN|ssl:10.0.249.214:45154: connection dropped (Protocol error)
2020-06-02T16:09:53Z|01129|raft|INFO|rejecting term 813 < current term 818 received in append_request message from server db04
2020-06-02T16:09:53Z|01130|raft|INFO|rejecting term 813 < current term 818 received in append_request message from server db04
2020-06-02T16:09:53Z|01131|raft|INFO|rejecting term 813 < current term 818 received in append_request message from server db04
2020-06-02T16:09:53Z|01132|raft|INFO|rejecting term 814 < current term 818 received in vote_request message from server db04
2020-06-02T16:09:53Z|01133|raft|INFO|rejecting term 816 < current term 818 received in vote_request message from server db04
2020-06-02T16:09:53Z|01134|raft|INFO|rejecting term 816 < current term 818 received in vote_reply message from server db04
2020-06-02T16:09:55Z|01135|raft|INFO|server db04 is leader for term 819
2020-06-02T16:09:55Z|01136|raft|INFO|rejecting append_request because previous entry 815,85374 not in local log (mismatch past end of log)
2020-06-02T16:09:55Z|01137|raft|INFO|rejecting append_request because previous entry 815,85374 not in local log (mismatch past end of log)
2020-06-02T16:09:55Z|01138|raft|INFO|rejecting append_request because previous entry 819,85375 not in local log (mismatch past end of log)
2020-06-02T16:09:55Z|01139|poll_loop|INFO|Dropped 708 log messages in last 5 seconds (most recently, 0 seconds ago) due to excessive rate
2020-06-02T16:09:55Z|01140|poll_loop|INFO|wakeup due to [POLLIN] on fd 13 (0.0.0.0:9642<->) at lib/stream-ssl.c:963 (100% CPU usage)
2020-06-02T16:09:56Z|01141|stream_ssl|WARN|Dropped 9 log messages in last 7 seconds (most recently, 2 seconds ago) due to excessive rate
2020-06-02T16:09:56Z|01142|stream_ssl|WARN|SSL_write: system error (Broken pipe)
2020-06-02T16:09:56Z|01143|reconnect|WARN|ssl:10.0.131.3:49398: connection dropped (Broken pipe)
2020-06-02T16:09:56Z|01144|raft|WARN|Dropped 3 log messages in last 18 seconds (most recently, 17 seconds ago) due to excessive rate
2020-06-02T16:09:56Z|01145|raft|WARN|ignoring vote request received after only 3 ms (minimum election time is 1000 ms)
2020-06-02T16:09:56Z|01146|raft|INFO|rejecting term 818 < current term 819 received in vote_reply message from server 77d9
2020-06-02T16:09:56Z|01147|reconnect|WARN|ssl:10.0.135.159:57616: connection dropped (Protocol error)
2020-06-02T16:09:57Z|01148|reconnect|WARN|ssl:10.0.133.115:43748: connection dropped (Input/output error)
2020-06-02T16:09:59Z|01149|reconnect|WARN|ssl:10.0.135.159:57888: connection dropped (Protocol error)
2020-06-02T16:10:03Z|01150|timeval|WARN|Unreasonably long 3901ms poll interval (3874ms user, 1ms system)
2020-06-02T16:10:03Z|01151|timeval|WARN|context switches: 0 voluntary, 3 involuntary
2020-06-02T16:10:03Z|01152|poll_loop|INFO|Dropped 285 log messages in last 8 seconds (most recently, 7 seconds ago) due to excessive rate
2020-06-02T16:10:03Z|01153|poll_loop|INFO|wakeup due to [POLLIN] on fd 13 (0.0.0.0:9642<->) at lib/stream-ssl.c:963 (86% CPU usage)
2020-06-02T16:10:03Z|01154|raft|INFO|90d3 received "operation still in progress" reply from db04 for unknown command
2020-06-02T16:10:03Z|01155|raft|INFO|90d3 received "operation still in progress" reply from db04 for unknown command
2020-06-02T16:10:03Z|01156|raft|INFO|90d3 received "operation still in progress" reply from db04 for unknown command
2020-06-02T16:10:03Z|01157|raft|WARN|ignoring vote request received after only 0 ms (minimum election time is 1000 ms)
2020-06-02T16:10:03Z|01158|stream_ssl|WARN|Dropped 4 log messages in last 6 seconds (most recently, 4 seconds ago) due to excessive rate
2020-06-02T16:10:03Z|01159|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:10:03Z|01160|jsonrpc|WARN|Dropped 13 log messages in last 14 seconds (most recently, 4 seconds ago) due to excessive rate
2020-06-02T16:10:03Z|01161|jsonrpc|WARN|ssl:10.0.221.58:36202: receive error: Protocol error
2020-06-02T16:10:03Z|01162|reconnect|WARN|ssl:10.0.221.58:36202: connection dropped (Protocol error)
2020-06-02T16:10:04Z|01163|raft|INFO|server db04 is leader for term 821
2020-06-02T16:10:04Z|01164|raft|INFO|ssl:10.0.135.159:57970: learned server ID 77d9
2020-06-02T16:10:04Z|01165|raft|INFO|ssl:10.0.135.159:57970: learned remote address ssl:10.0.135.159:9644
2020-06-02T16:10:04Z|01166|raft|INFO|rejecting term 819 < current term 821 received in vote_reply message from server 77d9
2020-06-02T16:10:11Z|01167|timeval|WARN|Unreasonably long 4552ms poll interval (4435ms user, 58ms system)
2020-06-02T16:10:11Z|01168|timeval|WARN|faults: 53 minor, 0 major
2020-06-02T16:10:11Z|01169|timeval|WARN|disk: 0 reads, 8 writes
2020-06-02T16:10:11Z|01170|timeval|WARN|context switches: 0 voluntary, 5 involuntary
2020-06-02T16:10:11Z|01171|poll_loop|INFO|Dropped 241 log messages in last 8 seconds (most recently, 5 seconds ago) due to excessive rate
2020-06-02T16:10:11Z|01172|poll_loop|INFO|wakeup due to [POLLIN] on fd 13 (0.0.0.0:9642<->) at lib/stream-ssl.c:963 (91% CPU usage)
2020-06-02T16:10:11Z|01173|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:10:11Z|01174|raft|INFO|Dropped 1 log messages in last 8 seconds (most recently, 8 seconds ago) due to excessive rate
2020-06-02T16:10:11Z|01175|raft|INFO|90d3 received "prerequisite check failed" reply from db04 for unknown command
2020-06-02T16:10:11Z|01176|raft|INFO|rejecting term 821 < current term 822 received in append_request message from server db04
2020-06-02T16:10:11Z|01177|raft|INFO|rejecting term 821 < current term 822 received in append_request message from server db04
2020-06-02T16:10:11Z|01178|reconnect|WARN|ssl:10.0.173.114:59110: connection dropped (Protocol error)
2020-06-02T16:10:11Z|01179|reconnect|WARN|ssl:10.0.173.114:59380: connection dropped (Protocol error)
2020-06-02T16:10:11Z|01180|raft|WARN|Dropped 2 log messages in last 0 seconds (most recently, 0 seconds ago) due to excessive rate
2020-06-02T16:10:11Z|01181|raft|WARN|ignoring vote request received after only 497 ms (minimum election time is 1000 ms)
2020-06-02T16:10:13Z|01182|raft|INFO|server 77d9 is leader for term 827
2020-06-02T16:10:13Z|01183|poll_loop|INFO|Dropped 292 log messages in last 2 seconds (most recently, 0 seconds ago) due to excessive rate
2020-06-02T16:10:13Z|01184|poll_loop|INFO|wakeup due to [POLLIN] on fd 38 (10.0.221.58:9642<->10.0.131.0:56488) at lib/stream-ssl.c:827 (91% CPU usage)
2020-06-02T16:10:20Z|01185|raft|INFO|Dropped 4 log messages in last 32 seconds (most recently, 17 seconds ago) due to excessive rate
2020-06-02T16:10:20Z|01186|raft|INFO|term 828: 5104 ms timeout expired, starting election
2020-06-02T16:10:20Z|01187|timeval|WARN|Unreasonably long 4537ms poll interval (4430ms user, 56ms system)
2020-06-02T16:10:20Z|01188|timeval|WARN|faults: 461 minor, 0 major
2020-06-02T16:10:20Z|01189|timeval|WARN|disk: 0 reads, 8 writes
2020-06-02T16:10:20Z|01190|timeval|WARN|context switches: 0 voluntary, 9 involuntary
2020-06-02T16:10:20Z|01191|coverage|INFO|Dropped 9 log messages in last 55 seconds (most recently, 10 seconds ago) due to excessive rate
2020-06-02T16:10:20Z|01192|coverage|INFO|Skipping details of duplicate event coverage for hash=1e253855
2020-06-02T16:10:20Z|01193|poll_loop|INFO|Dropped 24 log messages in last 8 seconds (most recently, 7 seconds ago) due to excessive rate
2020-06-02T16:10:20Z|01194|poll_loop|INFO|wakeup due to [POLLIN] on fd 13 (0.0.0.0:9642<->) at lib/stream-ssl.c:963 (75% CPU usage)
2020-06-02T16:10:20Z|01195|raft|INFO|rejecting term 826 < current term 828 received in append_request message from server db04
2020-06-02T16:10:20Z|01196|raft|INFO|rejecting term 826 < current term 828 received in append_request message from server db04
2020-06-02T16:10:20Z|01197|raft|INFO|Dropped 4 log messages in last 10 seconds (most recently, 10 seconds ago) due to excessive rate
2020-06-02T16:10:20Z|01198|raft|INFO|90d3 received "success" reply from 77d9 for unknown command
2020-06-02T16:10:20Z|01199|raft|INFO|rejecting term 827 < current term 828 received in append_request message from server 77d9
2020-06-02T16:10:20Z|01200|raft|INFO|rejecting term 827 < current term 828 received in append_request message from server 77d9
2020-06-02T16:10:20Z|01201|raft|INFO|rejecting term 827 < current term 828 received in append_request message from server 77d9
2020-06-02T16:10:20Z|01202|raft|INFO|rejecting term 827 < current term 828 received in append_request message from server 77d9
2020-06-02T16:10:20Z|01203|raft|INFO|rejecting term 827 < current term 828 received in append_request message from server 77d9
2020-06-02T16:10:20Z|01204|raft|INFO|rejecting term 827 < current term 828 received in append_request message from server 77d9
2020-06-02T16:10:20Z|01205|stream_ssl|WARN|Dropped 1 log messages in last 10 seconds (most recently, 10 seconds ago) due to excessive rate
2020-06-02T16:10:20Z|01206|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:10:20Z|01207|jsonrpc|WARN|Dropped 2 log messages in last 10 seconds (most recently, 10 seconds ago) due to excessive rate
2020-06-02T16:10:20Z|01208|jsonrpc|WARN|ssl:10.0.135.159:47376: receive error: Protocol error
2020-06-02T16:10:20Z|01209|reconnect|WARN|ssl:10.0.135.159:47376: connection dropped (Protocol error)
2020-06-02T16:10:22Z|01210|raft|INFO|rejecting term 827 < current term 829 received in append_request message from server 77d9
2020-06-02T16:10:22Z|01211|raft|INFO|rejecting term 827 < current term 829 received in vote_reply message from server 77d9
2020-06-02T16:10:23Z|01212|raft|INFO|ssl:10.0.135.159:58220: learned server ID 77d9
2020-06-02T16:10:23Z|01213|raft|INFO|ssl:10.0.135.159:58220: learned remote address ssl:10.0.135.159:9644
2020-06-02T16:10:24Z|01214|raft|INFO|server db04 is leader for term 831
2020-06-02T16:10:24Z|01215|raft|INFO|rejecting append_request because previous entry 827,85404 not in local log (mismatch past end of log)
2020-06-02T16:10:24Z|01216|raft|INFO|rejecting append_request because previous entry 827,85404 not in local log (mismatch past end of log)
2020-06-02T16:10:25Z|01217|poll_loop|INFO|Dropped 558 log messages in last 4 seconds (most recently, 0 seconds ago) due to excessive rate
2020-06-02T16:10:25Z|01218|poll_loop|INFO|wakeup due to [POLLIN] on fd 13 (0.0.0.0:9642<->) at lib/stream-ssl.c:963 (55% CPU usage)
2020-06-02T16:10:31Z|01219|stream_ssl|WARN|SSL_write: system error (Connection reset by peer)
2020-06-02T16:10:31Z|01220|jsonrpc|WARN|ssl:10.0.251.235:34924: send error: Connection reset by peer
2020-06-02T16:10:32Z|01221|reconnect|WARN|ssl:10.0.251.235:34924: connection dropped (Connection reset by peer)
2020-06-02T16:10:33Z|01222|poll_loop|INFO|Dropped 61 log messages in last 8 seconds (most recently, 7 seconds ago) due to excessive rate
2020-06-02T16:10:33Z|01223|poll_loop|INFO|wakeup due to [POLLOUT] on fd 47 (10.0.221.58:9642<->10.0.230.162:46170) at lib/stream-ssl.c:793 (101% CPU usage)
2020-06-02T16:10:36Z|01224|raft|INFO|server 77d9 is leader for term 832
2020-06-02T16:10:37Z|01225|poll_loop|INFO|Dropped 232 log messages in last 5 seconds (most recently, 1 seconds ago) due to excessive rate
2020-06-02T16:10:37Z|01226|poll_loop|INFO|wakeup due to [POLLOUT] on fd 75 (10.0.221.58:9642<->10.0.206.138:50876) at lib/stream-ssl.c:793 (101% CPU usage)
2020-06-02T16:10:38Z|01227|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:10:38Z|01228|jsonrpc|WARN|ssl:10.0.142.155:48894: receive error: Protocol error
2020-06-02T16:10:38Z|01229|timeval|WARN|Unreasonably long 1112ms poll interval (1103ms user, 1ms system)
2020-06-02T16:10:38Z|01230|timeval|WARN|context switches: 0 voluntary, 10 involuntary
2020-06-02T16:10:38Z|01231|reconnect|WARN|ssl:10.0.142.155:48894: connection dropped (Protocol error)
2020-06-02T16:10:38Z|01232|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:10:38Z|01233|raft|INFO|rejecting term 831 < current term 832 received in append_request message from server db04
2020-06-02T16:10:39Z|01234|reconnect|WARN|ssl:10.0.166.155:51378: connection dropped (Protocol error)
2020-06-02T16:10:39Z|01235|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:10:39Z|01236|reconnect|WARN|ssl:10.0.161.71:32812: connection dropped (Protocol error)
2020-06-02T16:10:39Z|01237|reconnect|WARN|ssl:10.0.173.20:40412: connection dropped (Protocol error)
2020-06-02T16:10:39Z|01238|reconnect|WARN|ssl:10.0.206.55:32988: connection dropped (Protocol error)
2020-06-02T16:10:39Z|01239|raft|WARN|ignoring vote request received after only 918 ms (minimum election time is 1000 ms)
2020-06-02T16:10:40Z|01240|raft|INFO|server db04 is leader for term 834
2020-06-02T16:10:42Z|01241|raft|INFO|rejecting term 832 < current term 834 received in append_request message from server 77d9
2020-06-02T16:10:42Z|01242|raft|INFO|rejecting term 832 < current term 834 received in append_request message from server 77d9
2020-06-02T16:10:53Z|01243|raft|INFO|term 835: elected leader by 2+ of 3 servers
2020-06-02T16:10:53Z|01244|raft|INFO|rejecting term 834 < current term 835 received in append_request message from server db04
2020-06-02T16:10:53Z|01245|raft|INFO|rejecting term 834 < current term 835 received in vote_reply message from server db04
2020-06-02T16:11:01Z|01246|raft|WARN|ignoring vote request received as leader
2020-06-02T16:11:01Z|01247|raft|WARN|ignoring vote request received as leader
2020-06-02T16:11:01Z|01248|timeval|WARN|Unreasonably long 5770ms poll interval (5698ms user, 33ms system)
2020-06-02T16:11:01Z|01249|timeval|WARN|faults: 2695 minor, 0 major
2020-06-02T16:11:01Z|01250|timeval|WARN|context switches: 0 voluntary, 57 involuntary
2020-06-02T16:11:01Z|01251|poll_loop|INFO|Dropped 272 log messages in last 23 seconds (most recently, 19 seconds ago) due to excessive rate
2020-06-02T16:11:01Z|01252|poll_loop|INFO|wakeup due to [POLLOUT] on fd 101 (10.0.221.58:9642<->10.0.129.0:55364) at lib/stream-ssl.c:793 (94% CPU usage)
2020-06-02T16:11:01Z|01253|poll_loop|INFO|wakeup due to [POLLOUT] on fd 59 (10.0.221.58:9642<->10.0.239.59:40774) at lib/stream-ssl.c:793 (94% CPU usage)
2020-06-02T16:11:01Z|01254|poll_loop|INFO|wakeup due to [POLLOUT] on fd 37 (10.0.221.58:9642<->10.0.169.188:34216) at lib/stream-ssl.c:793 (94% CPU usage)
2020-06-02T16:11:01Z|01255|stream_ssl|WARN|Dropped 2 log messages in last 22 seconds (most recently, 22 seconds ago) due to excessive rate
2020-06-02T16:11:01Z|01256|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:11:01Z|01257|jsonrpc|WARN|Dropped 4 log messages in last 22 seconds (most recently, 22 seconds ago) due to excessive rate
2020-06-02T16:11:01Z|01258|jsonrpc|WARN|ssl:10.0.135.159:47864: receive error: Protocol error
2020-06-02T16:11:01Z|01259|raft|WARN|ignoring vote request received as leader
2020-06-02T16:11:01Z|01260|reconnect|WARN|ssl:10.0.135.159:47864: connection dropped (Protocol error)
2020-06-02T16:11:01Z|01261|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:11:01Z|01262|jsonrpc|WARN|ssl:10.0.173.114:34222: receive error: Protocol error
2020-06-02T16:11:01Z|01263|raft|INFO|rejected append_reply (not leader)
2020-06-02T16:11:01Z|01264|reconnect|WARN|ssl:10.0.173.114:34222: connection dropped (Protocol error)
2020-06-02T16:11:01Z|01265|poll_loop|INFO|Dropped 219 log messages in last 1 seconds (most recently, 1 seconds ago) due to excessive rate
2020-06-02T16:11:01Z|01266|poll_loop|INFO|wakeup due to [POLLOUT] on fd 101 (10.0.221.58:9642<->10.0.129.0:55364) at lib/stream-ssl.c:793 (94% CPU usage)
2020-06-02T16:11:02Z|01267|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:11:02Z|01268|reconnect|WARN|ssl:10.0.221.58:36770: connection dropped (Protocol error)
2020-06-02T16:11:02Z|01269|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:11:03Z|01270|reconnect|WARN|ssl:10.0.173.114:34292: connection dropped (Protocol error)
2020-06-02T16:11:03Z|01271|reconnect|WARN|ssl:10.0.135.159:47920: connection dropped (Protocol error)
2020-06-02T16:11:03Z|01272|raft|INFO|server db04 is leader for term 840
2020-06-02T16:11:03Z|01273|raft|INFO|rejecting term 835 < current term 840 received in append_reply message from server 77d9
2020-06-02T16:11:03Z|01274|raft|INFO|rejecting term 835 < current term 840 received in append_reply message from server 77d9
2020-06-02T16:11:03Z|01275|raft|INFO|rejecting term 835 < current term 840 received in append_reply message from server 77d9
2020-06-02T16:11:03Z|01276|reconnect|WARN|ssl:10.0.221.58:36792: connection dropped (Protocol error)
2020-06-02T16:11:03Z|01277|raft|INFO|rejected append_reply (not leader)
2020-06-02T16:11:03Z|01278|reconnect|WARN|ssl:10.0.173.114:34478: connection dropped (Protocol error)
2020-06-02T16:11:07Z|01279|poll_loop|INFO|Dropped 435 log messages in last 5 seconds (most recently, 0 seconds ago) due to excessive rate
2020-06-02T16:11:07Z|01280|poll_loop|INFO|wakeup due to [POLLIN] on fd 26 (10.0.221.58:9644<->10.0.173.114:59758) at lib/stream-ssl.c:827 (101% CPU usage)
2020-06-02T16:11:21Z|01281|raft|INFO|Dropped 4 log messages in last 60 seconds (most recently, 29 seconds ago) due to excessive rate
2020-06-02T16:11:21Z|01282|raft|INFO|term 841: 1601 ms timeout expired, starting election
2020-06-02T16:11:21Z|01283|raft|INFO|term 841: elected leader by 2+ of 3 servers
2020-06-02T16:11:25Z|01284|raft|INFO|rejecting term 840 < current term 841 received in append_request message from server db04
2020-06-02T16:11:25Z|01285|raft|INFO|rejecting term 840 < current term 841 received in vote_reply message from server db04
2020-06-02T16:11:45Z|01286|stream_ssl|WARN|Dropped 3 log messages in last 42 seconds (most recently, 41 seconds ago) due to excessive rate
2020-06-02T16:11:45Z|01287|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:11:45Z|01288|jsonrpc|WARN|Dropped 5 log messages in last 43 seconds (most recently, 41 seconds ago) due to excessive rate
2020-06-02T16:11:45Z|01289|jsonrpc|WARN|ssl:10.0.135.159:48480: receive error: Protocol error
2020-06-02T16:11:45Z|01290|raft|WARN|Dropped 1 log messages in last 44 seconds (most recently, 44 seconds ago) due to excessive rate
2020-06-02T16:11:45Z|01291|raft|WARN|ignoring vote request received as leader
2020-06-02T16:11:45Z|01292|raft|WARN|ignoring vote request received as leader
2020-06-02T16:11:45Z|01293|timeval|WARN|Unreasonably long 2567ms poll interval (2502ms user, 45ms system)
2020-06-02T16:11:45Z|01294|timeval|WARN|context switches: 0 voluntary, 22 involuntary
2020-06-02T16:11:45Z|01295|coverage|INFO|Dropped 2 log messages in last 66 seconds (most recently, 44 seconds ago) due to excessive rate
2020-06-02T16:11:45Z|01296|coverage|INFO|Skipping details of duplicate event coverage for hash=1e253855
2020-06-02T16:11:45Z|01297|reconnect|WARN|ssl:10.0.135.159:48480: connection dropped (Protocol error)
2020-06-02T16:11:45Z|01298|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:11:45Z|01299|jsonrpc|WARN|ssl:10.0.173.114:39398: receive error: Protocol error
2020-06-02T16:11:45Z|01300|raft|INFO|rejected append_reply (not leader)
2020-06-02T16:11:45Z|01301|raft|WARN|ignoring vote request received after only 0 ms (minimum election time is 1000 ms)
2020-06-02T16:11:45Z|01302|raft|INFO|rejected append_reply (not leader)
2020-06-02T16:11:45Z|01303|reconnect|WARN|ssl:10.0.173.114:39398: connection dropped (Protocol error)
2020-06-02T16:11:45Z|01304|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:11:45Z|01305|jsonrpc|WARN|ssl:10.0.155.194:47592: receive error: Protocol error
2020-06-02T16:11:45Z|01306|raft|INFO|rejected append_reply (not leader)
2020-06-02T16:11:45Z|01307|raft|INFO|rejected append_reply (not leader)
2020-06-02T16:11:45Z|01308|reconnect|WARN|ssl:10.0.155.194:47592: connection dropped (Protocol error)
2020-06-02T16:11:45Z|01309|poll_loop|INFO|wakeup due to [POLLIN] on fd 26 (10.0.221.58:9644<->10.0.173.114:59758) at lib/stream-ssl.c:827 (90% CPU usage)
2020-06-02T16:11:45Z|01310|poll_loop|INFO|wakeup due to [POLLIN] on fd 20 (FIFO pipe:[103545371]) at ovsdb/log.c:994 (90% CPU usage)
2020-06-02T16:11:45Z|01311|poll_loop|INFO|wakeup due to [POLLIN] on fd 26 (10.0.221.58:9644<->10.0.173.114:59758) at lib/stream-ssl.c:827 (90% CPU usage)
2020-06-02T16:11:45Z|01312|raft|INFO|server db04 is leader for term 845
2020-06-02T16:11:45Z|01313|poll_loop|INFO|wakeup due to [POLLIN] on fd 20 (FIFO pipe:[103545371]) at ovsdb/log.c:994 (90% CPU usage)
2020-06-02T16:11:45Z|01314|poll_loop|INFO|wakeup due to [POLLIN] on fd 20 (FIFO pipe:[103545371]) at ovsdb/log.c:994 (90% CPU usage)
2020-06-02T16:11:45Z|01315|poll_loop|INFO|wakeup due to [POLLIN] on fd 26 (10.0.221.58:9644<->10.0.173.114:59758) at lib/stream-ssl.c:827 (90% CPU usage)
2020-06-02T16:11:56Z|01316|timeval|WARN|Unreasonably long 6242ms poll interval (5747ms user, 193ms system)
2020-06-02T16:11:56Z|01317|timeval|WARN|faults: 29013 minor, 0 major
2020-06-02T16:11:56Z|01318|timeval|WARN|disk: 0 reads, 318176 writes
2020-06-02T16:11:56Z|01319|timeval|WARN|context switches: 994 voluntary, 16 involuntary
2020-06-02T16:11:56Z|01320|poll_loop|INFO|Dropped 50 log messages in last 11 seconds (most recently, 9 seconds ago) due to excessive rate
2020-06-02T16:11:56Z|01321|poll_loop|INFO|wakeup due to [POLLIN] on fd 13 (0.0.0.0:9642<->) at lib/stream-ssl.c:963 (72% CPU usage)
2020-06-02T16:11:56Z|01322|poll_loop|INFO|wakeup due to [POLLIN] on fd 14 (/var/run/ovn/ovnsb_db.ctl<->) at lib/stream-fd.c:274 (72% CPU usage)
2020-06-02T16:11:56Z|01323|jsonrpc|WARN|unix#251: send error: Broken pipe
2020-06-02T16:11:58Z|01324|raft|INFO|rejecting term 845 < current term 846 received in vote_reply message from server 77d9
2020-06-02T16:11:59Z|01325|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:11:59Z|01326|jsonrpc|WARN|ssl:10.0.221.58:37348: receive error: Protocol error
2020-06-02T16:12:00Z|01327|reconnect|WARN|ssl:10.0.221.58:37348: connection dropped (Protocol error)
2020-06-02T16:12:00Z|01328|stream_ssl|WARN|SSL_write: system error (Broken pipe)
2020-06-02T16:12:00Z|01329|stream_ssl|WARN|SSL_write: error:1409E10F:SSL routines:ssl3_write_bytes:bad length
2020-06-02T16:12:00Z|01330|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:12:00Z|01331|raft|INFO|rejecting term 845 < current term 847 received in vote_reply message from server 77d9
2020-06-02T16:12:00Z|01332|reconnect|WARN|ssl:10.0.241.187:34208: connection dropped (Input/output error)
2020-06-02T16:12:00Z|01333|reconnect|WARN|ssl:10.0.241.34:53454: connection dropped (Protocol error)
2020-06-02T16:12:00Z|01334|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:12:00Z|01335|reconnect|WARN|ssl:10.0.225.111:47758: connection dropped (Protocol error)
2020-06-02T16:12:00Z|01336|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:12:00Z|01337|reconnect|WARN|ssl:10.0.228.42:59272: connection dropped (Protocol error)
2020-06-02T16:12:00Z|01338|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:12:00Z|01339|reconnect|WARN|ssl:10.0.243.94:39264: connection dropped (Protocol error)
2020-06-02T16:12:00Z|01340|reconnect|WARN|ssl:10.0.248.117:58126: connection dropped (Protocol error)
2020-06-02T16:12:00Z|01341|reconnect|WARN|ssl:10.0.213.32:52362: connection dropped (Protocol error)
2020-06-02T16:12:00Z|01342|reconnect|WARN|ssl:10.0.219.89:48736: connection dropped (Protocol error)
2020-06-02T16:12:00Z|01343|reconnect|WARN|ssl:10.0.183.243:51072: connection dropped (Protocol error)
2020-06-02T16:12:00Z|01344|reconnect|WARN|ssl:10.0.192.128:35744: connection dropped (Protocol error)
2020-06-02T16:12:00Z|01345|raft|INFO|rejecting term 845 < current term 847 received in append_request message from server db04
2020-06-02T16:12:00Z|01346|raft|INFO|rejecting term 845 < current term 847 received in append_request message from server db04
2020-06-02T16:12:00Z|01347|raft|INFO|rejecting term 845 < current term 847 received in append_request message from server db04
2020-06-02T16:12:00Z|01348|raft|INFO|rejecting term 845 < current term 847 received in append_request message from server db04
2020-06-02T16:12:00Z|01349|raft|INFO|rejecting term 845 < current term 847 received in append_request message from server db04
2020-06-02T16:12:00Z|01350|raft|INFO|rejecting term 845 < current term 847 received in vote_reply message from server db04
2020-06-02T16:12:00Z|01351|raft|INFO|rejecting term 845 < current term 847 received in append_request message from server db04
2020-06-02T16:12:00Z|01352|raft|INFO|rejecting term 845 < current term 847 received in append_request message from server db04
2020-06-02T16:12:00Z|01353|raft|INFO|rejecting term 845 < current term 847 received in append_request message from server db04
2020-06-02T16:12:00Z|01354|raft|INFO|rejecting term 845 < current term 847 received in append_request message from server db04
2020-06-02T16:12:00Z|01355|raft|INFO|rejecting term 845 < current term 847 received in append_request message from server db04
2020-06-02T16:12:00Z|01356|raft|INFO|rejecting term 845 < current term 847 received in append_request message from server db04
2020-06-02T16:12:00Z|01357|raft|INFO|rejecting term 845 < current term 847 received in append_request message from server db04
2020-06-02T16:12:00Z|01358|raft|INFO|rejecting term 845 < current term 847 received in vote_reply message from server db04
2020-06-02T16:12:01Z|01359|raft|INFO|server 77d9 is leader for term 849
2020-06-02T16:12:01Z|01360|raft|INFO|rejecting append_request because previous entry 845,85539 not in local log (mismatch past end of log)
2020-06-02T16:12:01Z|01361|raft|INFO|rejecting append_request because previous entry 845,85539 not in local log (mismatch past end of log)
2020-06-02T16:12:01Z|01362|raft|INFO|rejecting append_request because previous entry 849,85540 not in local log (mismatch past end of log)
2020-06-02T16:12:01Z|01363|poll_loop|INFO|Dropped 368 log messages in last 4 seconds (most recently, 0 seconds ago) due to excessive rate
2020-06-02T16:12:01Z|01364|poll_loop|INFO|wakeup due to [POLLIN] on fd 25 (10.0.221.58:9644<->10.0.135.159:58220) at lib/stream-ssl.c:827 (99% CPU usage)
2020-06-02T16:12:07Z|01365|stream_ssl|WARN|Dropped 6 log messages in last 7 seconds (most recently, 5 seconds ago) due to excessive rate
2020-06-02T16:12:07Z|01366|stream_ssl|WARN|SSL_write: system error (Connection reset by peer)
2020-06-02T16:12:07Z|01367|reconnect|WARN|ssl:10.0.240.18:33218: connection dropped (Input/output error)
2020-06-02T16:12:08Z|01368|reconnect|WARN|ssl:10.0.163.107:42202: connection dropped (Input/output error)
2020-06-02T16:12:09Z|01369|reconnect|WARN|ssl:10.0.242.71:58522: connection dropped (Input/output error)
2020-06-02T16:12:09Z|01370|reconnect|WARN|ssl:10.0.191.74:37562: connection dropped (Input/output error)
2020-06-02T16:12:10Z|01371|reconnect|WARN|ssl:10.0.171.71:56738: connection dropped (Input/output error)
2020-06-02T16:12:10Z|01372|reconnect|WARN|ssl:10.0.160.112:34118: connection dropped (Input/output error)
2020-06-02T16:12:42Z|01373|timeval|WARN|Unreasonably long 1227ms poll interval (1208ms user, 6ms system)
2020-06-02T16:12:42Z|01374|timeval|WARN|disk: 0 reads, 16 writes
2020-06-02T16:12:42Z|01375|timeval|WARN|context switches: 0 voluntary, 2 involuntary
2020-06-02T16:12:42Z|01376|coverage|INFO|Dropped 1 log messages in last 45 seconds (most recently, 45 seconds ago) due to excessive rate
2020-06-02T16:12:42Z|01377|coverage|INFO|Skipping details of duplicate event coverage for hash=1e253855
2020-06-02T16:12:43Z|01378|poll_loop|INFO|Dropped 487 log messages in last 43 seconds (most recently, 38 seconds ago) due to excessive rate
2020-06-02T16:12:43Z|01379|poll_loop|INFO|wakeup due to [POLLOUT] on fd 27 (10.0.221.58:9642<->10.0.249.214:45930) at lib/stream-ssl.c:793 (99% CPU usage)
2020-06-02T16:12:43Z|01380|poll_loop|INFO|wakeup due to [POLLOUT] on fd 33 (10.0.221.58:9642<->10.0.241.34:53676) at lib/stream-ssl.c:793 (99% CPU usage)
2020-06-02T16:12:43Z|01381|poll_loop|INFO|wakeup due to [POLLOUT] on fd 28 (10.0.221.58:9642<->10.0.136.48:33448) at lib/stream-ssl.c:793 (99% CPU usage)
2020-06-02T16:12:43Z|01382|poll_loop|INFO|wakeup due to [POLLOUT] on fd 30 (10.0.221.58:9642<->10.0.254.230:52854) at lib/stream-ssl.c:793 (99% CPU usage)
2020-06-02T16:12:43Z|01383|poll_loop|INFO|wakeup due to [POLLOUT] on fd 31 (10.0.221.58:9642<->10.0.243.175:55202) at lib/stream-ssl.c:793 (99% CPU usage)
2020-06-02T16:12:43Z|01384|poll_loop|INFO|wakeup due to [POLLOUT] on fd 32 (10.0.221.58:9642<->10.0.193.248:44248) at lib/stream-ssl.c:793 (99% CPU usage)
2020-06-02T16:12:43Z|01385|poll_loop|INFO|wakeup due to [POLLOUT] on fd 34 (10.0.221.58:9642<->10.0.188.0:53718) at lib/stream-ssl.c:793 (99% CPU usage)
2020-06-02T16:13:02Z|01386|raft|INFO|Dropped 3 log messages in last 64 seconds (most recently, 61 seconds ago) due to excessive rate
2020-06-02T16:13:02Z|01387|raft|INFO|term 850: 1499 ms timeout expired, starting election
2020-06-02T16:13:02Z|01388|raft|INFO|rejecting term 849 < current term 850 received in vote_reply message from server 77d9
2020-06-02T16:13:02Z|01389|raft|INFO|rejecting term 849 < current term 850 received in append_request message from server 77d9
2020-06-02T16:13:03Z|01390|raft|INFO|server db04 is leader for term 851
2020-06-02T16:13:16Z|01391|timeval|WARN|Unreasonably long 2535ms poll interval (2462ms user, 56ms system)
2020-06-02T16:13:16Z|01392|timeval|WARN|faults: 9471 minor, 0 major
2020-06-02T16:13:16Z|01393|timeval|WARN|context switches: 0 voluntary, 23 involuntary
2020-06-02T16:13:16Z|01394|poll_loop|INFO|Dropped 72 log messages in last 32 seconds (most recently, 29 seconds ago) due to excessive rate
2020-06-02T16:13:16Z|01395|poll_loop|INFO|wakeup due to [POLLOUT] on fd 59 (10.0.221.58:9642<->10.0.130.210:40372) at lib/stream-ssl.c:793 (77% CPU usage)
2020-06-02T16:13:16Z|01396|poll_loop|INFO|wakeup due to [POLLOUT] on fd 37 (10.0.221.58:9642<->10.0.173.20:41130) at lib/stream-ssl.c:793 (77% CPU usage)
2020-06-02T16:13:16Z|01397|poll_loop|INFO|wakeup due to [POLLOUT] on fd 27 (10.0.221.58:9642<->10.0.249.214:45930) at lib/stream-ssl.c:793 (77% CPU usage)
2020-06-02T16:13:16Z|01398|poll_loop|INFO|wakeup due to [POLLOUT] on fd 75 (10.0.221.58:9642<->10.0.161.71:33532) at lib/stream-ssl.c:793 (77% CPU usage)
2020-06-02T16:13:16Z|01399|poll_loop|INFO|wakeup due to [POLLOUT] on fd 47 (10.0.221.58:9642<->10.0.216.234:47704) at lib/stream-ssl.c:793 (77% CPU usage)
2020-06-02T16:13:16Z|01400|raft|INFO|rejecting term 851 < current term 852 received in append_request message from server db04
2020-06-02T16:13:16Z|01401|raft|INFO|rejecting term 851 < current term 852 received in append_request message from server db04
2020-06-02T16:13:16Z|01402|raft|INFO|rejecting term 851 < current term 852 received in append_request message from server db04
2020-06-02T16:13:16Z|01403|raft|INFO|rejecting term 851 < current term 852 received in append_request message from server db04
2020-06-02T16:13:16Z|01404|raft|INFO|rejecting term 851 < current term 852 received in append_request message from server db04
2020-06-02T16:13:17Z|01405|raft|INFO|server db04 is leader for term 853
2020-06-02T16:13:25Z|01406|poll_loop|INFO|Dropped 121 log messages in last 10 seconds (most recently, 7 seconds ago) due to excessive rate
2020-06-02T16:13:25Z|01407|poll_loop|INFO|wakeup due to [POLLOUT] on fd 59 (10.0.221.58:9642<->10.0.130.210:40372) at lib/stream-ssl.c:793 (75% CPU usage)
2020-06-02T16:13:25Z|01408|poll_loop|INFO|wakeup due to [POLLOUT] on fd 37 (10.0.221.58:9642<->10.0.173.20:41130) at lib/stream-ssl.c:793 (75% CPU usage)
2020-06-02T16:13:33Z|01409|raft|WARN|Dropped 2 log messages in last 108 seconds (most recently, 108 seconds ago) due to excessive rate
2020-06-02T16:13:33Z|01410|raft|WARN|ignoring vote request received after only 1 ms (minimum election time is 1000 ms)
2020-06-02T16:13:34Z|01411|raft|INFO|server db04 is leader for term 856
2020-06-02T16:13:46Z|01412|timeval|WARN|Unreasonably long 1989ms poll interval (1860ms user, 101ms system)
2020-06-02T16:13:46Z|01413|timeval|WARN|faults: 14813 minor, 0 major
2020-06-02T16:13:46Z|01414|timeval|WARN|context switches: 0 voluntary, 3 involuntary
2020-06-02T16:13:46Z|01415|coverage|INFO|Dropped 1 log messages in last 31 seconds (most recently, 31 seconds ago) due to excessive rate
2020-06-02T16:13:46Z|01416|coverage|INFO|Skipping details of duplicate event coverage for hash=1e253855
2020-06-02T16:13:47Z|01417|poll_loop|INFO|Dropped 384 log messages in last 21 seconds (most recently, 19 seconds ago) due to excessive rate
2020-06-02T16:13:47Z|01418|poll_loop|INFO|wakeup due to [POLLIN] on fd 13 (0.0.0.0:9642<->) at lib/stream-ssl.c:963 (91% CPU usage)
2020-06-02T16:13:47Z|01419|poll_loop|INFO|wakeup due to [POLLOUT] on fd 128 (10.0.221.58:9642<->10.0.198.203:41480) at lib/stream-ssl.c:793 (91% CPU usage)
2020-06-02T16:13:47Z|01420|poll_loop|INFO|wakeup due to [POLLOUT] on fd 129 (10.0.221.58:9642<->10.0.185.67:40238) at lib/stream-ssl.c:793 (91% CPU usage)
2020-06-02T16:13:49Z|01421|timeval|WARN|Unreasonably long 1379ms poll interval (1288ms user, 68ms system)
2020-06-02T16:13:49Z|01422|timeval|WARN|faults: 3420 minor, 0 major
2020-06-02T16:13:49Z|01423|timeval|WARN|context switches: 0 voluntary, 18 involuntary
2020-06-02T16:13:49Z|01424|poll_loop|INFO|Dropped 221 log messages in last 2 seconds (most recently, 0 seconds ago) due to excessive rate
2020-06-02T16:13:49Z|01425|poll_loop|INFO|wakeup due to [POLLOUT] on fd 129 (10.0.221.58:9642<->10.0.185.67:40238) at lib/stream-ssl.c:793 (91% CPU usage)
2020-06-02T16:13:50Z|01426|stream_ssl|WARN|Dropped 11 log messages in last 104 seconds (most recently, 101 seconds ago) due to excessive rate
2020-06-02T16:13:50Z|01427|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:13:50Z|01428|jsonrpc|WARN|Dropped 16 log messages in last 111 seconds (most recently, 101 seconds ago) due to excessive rate
2020-06-02T16:13:50Z|01429|jsonrpc|WARN|ssl:10.0.203.110:37344: receive error: Protocol error
2020-06-02T16:13:50Z|01430|reconnect|WARN|ssl:10.0.203.110:37344: connection dropped (Protocol error)
2020-06-02T16:13:50Z|01431|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:13:50Z|01432|jsonrpc|WARN|ssl:10.0.223.227:42346: receive error: Protocol error
2020-06-02T16:13:51Z|01433|reconnect|WARN|ssl:10.0.223.227:42346: connection dropped (Protocol error)
2020-06-02T16:13:51Z|01434|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:13:51Z|01435|jsonrpc|WARN|ssl:10.0.140.19:42696: receive error: Protocol error
2020-06-02T16:13:51Z|01436|reconnect|WARN|ssl:10.0.140.19:42696: connection dropped (Protocol error)
2020-06-02T16:13:51Z|01437|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:13:51Z|01438|jsonrpc|WARN|ssl:10.0.128.209:40704: receive error: Protocol error
2020-06-02T16:13:51Z|01439|reconnect|WARN|ssl:10.0.128.209:40704: connection dropped (Protocol error)
2020-06-02T16:20:13Z|01440|raft|INFO|Dropped 1 log messages in last 417 seconds (most recently, 417 seconds ago) due to excessive rate
2020-06-02T16:20:13Z|01441|raft|INFO|term 857: 1627 ms timeout expired, starting election
2020-06-02T16:20:13Z|01442|raft|INFO|term 857: elected leader by 2+ of 3 servers
2020-06-02T16:20:17Z|01443|reconnect|ERR|ssl:10.0.173.114:9644: no response to inactivity probe after 5 seconds, disconnecting
2020-06-02T16:20:17Z|01444|reconnect|INFO|ssl:10.0.173.114:9644: connection dropped
2020-06-02T16:20:18Z|01445|reconnect|INFO|ssl:10.0.173.114:9644: connecting...
2020-06-02T16:20:19Z|01446|reconnect|INFO|ssl:10.0.173.114:9644: connection attempt timed out
2020-06-02T16:20:19Z|01447|reconnect|INFO|ssl:10.0.173.114:9644: waiting 2 seconds before reconnect
2020-06-02T16:20:19Z|01448|raft|INFO|rejecting term 856 < current term 857 received in append_request message from server db04
2020-06-02T16:20:20Z|01449|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:20:20Z|01450|jsonrpc|WARN|ssl:10.0.135.159:55210: receive error: Protocol error
2020-06-02T16:20:20Z|01451|reconnect|WARN|ssl:10.0.135.159:55210: connection dropped (Protocol error)
2020-06-02T16:20:21Z|01452|reconnect|INFO|ssl:10.0.173.114:9644: connecting...
2020-06-02T16:20:21Z|01453|reconnect|INFO|ssl:10.0.173.114:9644: connected
2020-06-02T16:21:20Z|01454|raft|INFO|ssl:10.0.135.159:38524: learned server ID 77d9
2020-06-02T16:21:20Z|01455|raft|INFO|ssl:10.0.135.159:38524: learned remote address ssl:10.0.135.159:9644
2020-06-02T16:23:30Z|01456|timeval|WARN|Unreasonably long 7538ms poll interval (6757ms user, 316ms system)
2020-06-02T16:23:30Z|01457|timeval|WARN|faults: 92587 minor, 0 major
2020-06-02T16:23:30Z|01458|timeval|WARN|disk: 0 reads, 370496 writes
2020-06-02T16:23:30Z|01459|timeval|WARN|context switches: 1321 voluntary, 13 involuntary
2020-06-02T16:23:30Z|01460|coverage|INFO|Dropped 1 log messages in last 582 seconds (most recently, 582 seconds ago) due to excessive rate
2020-06-02T16:23:30Z|01461|coverage|INFO|Event coverage, avg rate over last: 5 seconds, last minute, last hour, hash=25b513c2:
2020-06-02T16:23:30Z|01462|coverage|INFO|hmap_pathological 0.6/sec 0.600/sec 5.0108/sec total: 18071
2020-06-02T16:23:30Z|01463|coverage|INFO|hmap_expand 98.0/sec 132.500/sec 6026.7781/sec total: 22176969
2020-06-02T16:23:30Z|01464|coverage|INFO|lockfile_lock 0.0/sec 0.000/sec 0.0003/sec total: 1
2020-06-02T16:23:30Z|01465|coverage|INFO|poll_create_node 1669.6/sec 2490.783/sec 2455.1153/sec total: 8841424
2020-06-02T16:23:30Z|01466|coverage|INFO|poll_zero_timeout 3.6/sec 4.917/sec 38.5361/sec total: 138736
2020-06-02T16:23:30Z|01467|coverage|INFO|seq_change 12.2/sec 18.483/sec 50.2100/sec total: 180781
2020-06-02T16:23:30Z|01468|coverage|INFO|pstream_open 0.0/sec 0.000/sec 0.0011/sec total: 4
2020-06-02T16:23:30Z|01469|coverage|INFO|stream_open 0.0/sec 0.000/sec 0.0014/sec total: 5
2020-06-02T16:23:30Z|01470|coverage|INFO|unixctl_received 0.2/sec 0.100/sec 0.0892/sec total: 321
2020-06-02T16:23:30Z|01471|coverage|INFO|unixctl_replied 0.2/sec 0.100/sec 0.0892/sec total: 321
2020-06-02T16:23:30Z|01472|coverage|INFO|util_xalloc 3736.6/sec 5896.800/sec 287487.8478/sec total: 1061824620
2020-06-02T16:23:30Z|01473|coverage|INFO|68 events never hit
2020-06-02T16:23:30Z|01474|poll_loop|INFO|Dropped 208 log messages in last 582 seconds (most recently, 578 seconds ago) due to excessive rate
2020-06-02T16:23:30Z|01475|poll_loop|INFO|wakeup due to [POLLIN] on fd 13 (0.0.0.0:9642<->) at lib/stream-ssl.c:963 (77% CPU usage)
2020-06-02T16:23:30Z|01476|poll_loop|INFO|wakeup due to [POLLIN] on fd 16 (10.0.221.58:9642<->10.0.221.58:41944) at lib/stream-ssl.c:827 (77% CPU usage)
2020-06-02T16:23:30Z|01477|poll_loop|INFO|wakeup due to [POLLIN] on fd 17 (10.0.221.58:38126<->10.0.135.159:9644) at lib/stream-ssl.c:827 (77% CPU usage)
2020-06-02T16:23:30Z|01478|poll_loop|INFO|wakeup due to [POLLIN] on fd 25 (10.0.221.58:9644<->10.0.135.159:38524) at lib/stream-ssl.c:827 (77% CPU usage)
2020-06-02T16:23:30Z|01479|poll_loop|INFO|wakeup due to [POLLIN] on fd 26 (10.0.221.58:9644<->10.0.173.114:59758) at lib/stream-ssl.c:827 (77% CPU usage)
2020-06-02T16:23:30Z|01480|poll_loop|INFO|wakeup due to [POLLIN] on fd 154 (10.0.221.58:36006<->10.0.173.114:9644) at lib/stream-ssl.c:827 (77% CPU usage)
2020-06-02T16:23:30Z|01481|poll_loop|INFO|wakeup due to [POLLIN] on fd 155 (10.0.221.58:9642<->10.0.135.159:55428) at lib/stream-ssl.c:827 (77% CPU usage)
2020-06-02T16:23:30Z|01482|poll_loop|INFO|wakeup due to [POLLIN] on fd 153 (10.0.221.58:9642<->10.0.173.114:56430) at lib/stream-ssl.c:827 (77% CPU usage)
2020-06-02T16:23:30Z|01483|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:23:30Z|01484|jsonrpc|WARN|ssl:10.0.173.114:34666: receive error: Protocol error
2020-06-02T16:23:30Z|01485|raft|WARN|ignoring vote request received as leader
2020-06-02T16:23:30Z|01486|raft|INFO|server 77d9 is leader for term 858
2020-06-02T16:23:30Z|01487|raft|INFO|90d3 truncating 1 entries from end of log
2020-06-02T16:23:30Z|01488|stream_ssl|WARN|SSL_write: system error (Broken pipe)
2020-06-02T16:23:30Z|01489|jsonrpc|WARN|ssl:10.0.135.159:38524: send error: Broken pipe
2020-06-02T16:23:30Z|01490|poll_loop|INFO|wakeup due to [POLLIN] on fd 13 (0.0.0.0:9642<->) at lib/stream-ssl.c:963 (77% CPU usage)
2020-06-02T16:23:30Z|01491|stream_ssl|WARN|SSL_write: system error (Broken pipe)
2020-06-02T16:23:30Z|01492|jsonrpc|WARN|ssl:10.0.221.58:41944: send error: Broken pipe
2020-06-02T16:23:30Z|01493|reconnect|WARN|ssl:10.0.173.114:34666: connection dropped (Protocol error)
2020-06-02T16:23:31Z|01494|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:23:31Z|01495|jsonrpc|WARN|ssl:10.0.173.114:34730: receive error: Protocol error
2020-06-02T16:23:31Z|01496|raft|INFO|rejected append_reply (not leader)
2020-06-02T16:23:31Z|01497|raft|INFO|rejected append_reply (not leader)
2020-06-02T16:23:31Z|01498|reconnect|WARN|ssl:10.0.135.159:38524: connection dropped (Broken pipe)
2020-06-02T16:23:31Z|01499|poll_loop|INFO|wakeup due to [POLLIN] on fd 13 (0.0.0.0:9642<->) at lib/stream-ssl.c:963 (77% CPU usage)
2020-06-02T16:23:31Z|01500|reconnect|WARN|ssl:10.0.221.58:41944: connection dropped (Broken pipe)
2020-06-02T16:23:31Z|01501|reconnect|WARN|ssl:10.0.173.114:34730: connection dropped (Protocol error)
2020-06-02T16:23:31Z|01502|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:23:31Z|01503|jsonrpc|WARN|ssl:10.0.173.114:34844: receive error: Protocol error
2020-06-02T16:23:31Z|01504|reconnect|WARN|ssl:10.0.173.114:34844: connection dropped (Protocol error)
2020-06-02T16:23:31Z|01505|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:23:31Z|01506|reconnect|WARN|ssl:10.0.221.58:43686: connection dropped (Protocol error)
2020-06-02T16:23:31Z|01507|raft|INFO|ssl:10.0.135.159:40174: learned server ID 77d9
2020-06-02T16:23:31Z|01508|raft|INFO|ssl:10.0.135.159:40174: learned remote address ssl:10.0.135.159:9644
2020-06-02T16:39:41Z|01509|reconnect|ERR|ssl:10.0.173.114:9644: no response to inactivity probe after 5 seconds, disconnecting
2020-06-02T16:39:41Z|01510|reconnect|INFO|ssl:10.0.173.114:9644: connection dropped
2020-06-02T16:39:41Z|01511|reconnect|ERR|ssl:10.0.173.114:59758: no response to inactivity probe after 5 seconds, disconnecting
2020-06-02T16:39:42Z|01512|reconnect|INFO|ssl:10.0.173.114:9644: connecting...
2020-06-02T16:39:43Z|01513|reconnect|INFO|ssl:10.0.173.114:9644: connected
2020-06-02T16:39:44Z|01514|raft|INFO|ssl:10.0.173.114:53774: learned server ID db04
2020-06-02T16:39:44Z|01515|raft|INFO|ssl:10.0.173.114:53774: learned remote address ssl:10.0.173.114:9644
2020-06-02T16:40:34Z|01516|raft|INFO|server db04 is leader for term 859
2020-06-02T16:41:49Z|01517|timeval|WARN|Unreasonably long 7663ms poll interval (7065ms user, 147ms system)
2020-06-02T16:41:49Z|01518|timeval|WARN|faults: 93 minor, 0 major
2020-06-02T16:41:49Z|01519|timeval|WARN|disk: 0 reads, 370496 writes
2020-06-02T16:41:49Z|01520|timeval|WARN|context switches: 1872 voluntary, 66 involuntary
2020-06-02T16:41:49Z|01521|coverage|INFO|Skipping details of duplicate event coverage for hash=25b513c2
2020-06-02T16:41:49Z|01522|poll_loop|INFO|Dropped 23 log messages in last 1098 seconds (most recently, 1095 seconds ago) due to excessive rate
2020-06-02T16:41:49Z|01523|poll_loop|INFO|wakeup due to [POLLIN] on fd 13 (0.0.0.0:9642<->) at lib/stream-ssl.c:963 (86% CPU usage)
2020-06-02T16:41:49Z|01524|poll_loop|INFO|wakeup due to [POLLIN] on fd 14 (/var/run/ovn/ovnsb_db.ctl<->) at lib/stream-fd.c:274 (86% CPU usage)
2020-06-02T16:41:49Z|01525|poll_loop|INFO|wakeup due to [POLLIN] on fd 16 (10.0.221.58:9644<->10.0.135.159:40174) at lib/stream-ssl.c:827 (86% CPU usage)
2020-06-02T16:41:49Z|01526|poll_loop|INFO|wakeup due to [POLLIN] on fd 17 (10.0.221.58:38126<->10.0.135.159:9644) at lib/stream-ssl.c:827 (86% CPU usage)
2020-06-02T16:41:49Z|01527|poll_loop|INFO|wakeup due to [POLLIN] on fd 25 (10.0.221.58:46638<->10.0.173.114:9644) at lib/stream-ssl.c:827 (86% CPU usage)
2020-06-02T16:41:49Z|01528|poll_loop|INFO|wakeup due to [POLLIN] on fd 26 (10.0.221.58:9644<->10.0.173.114:53774) at lib/stream-ssl.c:827 (86% CPU usage)
2020-06-02T16:41:49Z|01529|jsonrpc|WARN|Dropped 1 log messages in last 1098 seconds (most recently, 1098 seconds ago) due to excessive rate
2020-06-02T16:41:49Z|01530|jsonrpc|WARN|unix#430: send error: Broken pipe
2020-06-02T16:41:49Z|01531|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:41:49Z|01532|jsonrpc|WARN|ssl:10.0.135.159:45200: receive error: Protocol error
2020-06-02T16:41:49Z|01533|poll_loop|INFO|wakeup due to [POLLIN] on fd 27 (10.0.221.58:9642<->10.0.249.214:45930) at lib/stream-ssl.c:827 (86% CPU usage)
2020-06-02T16:41:49Z|01534|poll_loop|INFO|wakeup due to [POLLIN] on fd 13 (0.0.0.0:9642<->) at lib/stream-ssl.c:963 (86% CPU usage)
2020-06-02T16:41:49Z|01535|poll_loop|INFO|wakeup due to [POLLIN] on fd 98 (10.0.221.58:9642<->10.0.225.111:48232) at lib/stream-ssl.c:827 (86% CPU usage)
2020-06-02T16:41:49Z|01536|poll_loop|INFO|wakeup due to [POLLIN] on fd 128 (10.0.221.58:9642<->10.0.198.203:41480) at lib/stream-ssl.c:827 (86% CPU usage)
2020-06-02T16:41:49Z|01537|reconnect|WARN|ssl:10.0.135.159:45200: connection dropped (Protocol error)
2020-06-02T16:41:49Z|01538|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:41:49Z|01539|jsonrpc|WARN|ssl:10.0.173.114:38676: receive error: Protocol error
2020-06-02T16:41:49Z|01540|reconnect|WARN|ssl:10.0.173.114:38676: connection dropped (Protocol error)
2020-06-02T16:41:50Z|01541|raft|INFO|ssl:10.0.173.114:57238: learned server ID db04
2020-06-02T16:41:50Z|01542|raft|INFO|ssl:10.0.173.114:57238: learned remote address ssl:10.0.173.114:9644
2020-06-02T16:56:59Z|01543|reconnect|ERR|ssl:10.0.135.159:9644: no response to inactivity probe after 5 seconds, disconnecting
2020-06-02T16:56:59Z|01544|reconnect|INFO|ssl:10.0.135.159:9644: connection dropped
2020-06-02T16:56:59Z|01545|reconnect|ERR|ssl:10.0.135.159:40174: no response to inactivity probe after 5 seconds, disconnecting
2020-06-02T16:57:00Z|01546|reconnect|INFO|ssl:10.0.135.159:9644: connecting...
2020-06-02T16:57:01Z|01547|reconnect|INFO|ssl:10.0.135.159:9644: connected
2020-06-02T16:57:02Z|01548|raft|INFO|ssl:10.0.135.159:40466: learned server ID 77d9
2020-06-02T16:57:02Z|01549|raft|INFO|ssl:10.0.135.159:40466: learned remote address ssl:10.0.135.159:9644
2020-06-02T16:58:37Z|01550|timeval|WARN|Unreasonably long 7814ms poll interval (7197ms user, 161ms system)
2020-06-02T16:58:37Z|01551|timeval|WARN|faults: 93 minor, 0 major
2020-06-02T16:58:37Z|01552|timeval|WARN|disk: 0 reads, 370496 writes
2020-06-02T16:58:37Z|01553|timeval|WARN|context switches: 1097 voluntary, 12 involuntary
2020-06-02T16:58:37Z|01554|coverage|INFO|Skipping details of duplicate event coverage for hash=25b513c2
2020-06-02T16:58:37Z|01555|poll_loop|INFO|Dropped 55 log messages in last 1008 seconds (most recently, 1005 seconds ago) due to excessive rate
2020-06-02T16:58:37Z|01556|poll_loop|INFO|wakeup due to [POLLIN] on fd 14 (/var/run/ovn/ovnsb_db.ctl<->) at lib/stream-fd.c:274 (86% CPU usage)
2020-06-02T16:58:37Z|01557|poll_loop|INFO|wakeup due to [POLLIN] on fd 16 (10.0.221.58:40176<->10.0.135.159:9644) at lib/stream-ssl.c:827 (86% CPU usage)
2020-06-02T16:58:37Z|01558|poll_loop|INFO|wakeup due to [POLLIN] on fd 15 (10.0.221.58:9644<->) at lib/stream-ssl.c:963 (86% CPU usage)
2020-06-02T16:58:37Z|01559|poll_loop|INFO|wakeup due to [POLLIN] on fd 17 (10.0.221.58:9644<->10.0.135.159:40466) at lib/stream-ssl.c:827 (86% CPU usage)
2020-06-02T16:58:37Z|01560|poll_loop|INFO|wakeup due to [POLLIN] on fd 25 (10.0.221.58:46638<->10.0.173.114:9644) at lib/stream-ssl.c:827 (86% CPU usage)
2020-06-02T16:58:37Z|01561|poll_loop|INFO|wakeup due to [POLLIN] on fd 26 (10.0.221.58:9644<->10.0.173.114:57238) at lib/stream-ssl.c:827 (86% CPU usage)
2020-06-02T16:58:37Z|01562|jsonrpc|WARN|unix#531: send error: Broken pipe
2020-06-02T16:58:37Z|01563|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T16:58:37Z|01564|jsonrpc|WARN|ssl:10.0.173.114:60982: receive error: Protocol error
2020-06-02T16:58:37Z|01565|poll_loop|INFO|wakeup due to [POLLIN] on fd 59 (10.0.221.58:9642<->10.0.130.210:40372) at lib/stream-ssl.c:827 (86% CPU usage)
2020-06-02T16:58:37Z|01566|poll_loop|INFO|wakeup due to [POLLIN] on fd 37 (10.0.221.58:9642<->10.0.173.20:41130) at lib/stream-ssl.c:827 (86% CPU usage)
2020-06-02T16:58:37Z|01567|poll_loop|INFO|wakeup due to [POLLIN] on fd 47 (10.0.221.58:9642<->10.0.216.234:47704) at lib/stream-ssl.c:827 (86% CPU usage)
2020-06-02T16:58:37Z|01568|poll_loop|INFO|wakeup due to [POLLIN] on fd 66 (10.0.221.58:9642<->10.0.133.213:50188) at lib/stream-ssl.c:827 (86% CPU usage)
2020-06-02T16:58:37Z|01569|raft|INFO|rejecting append_request because previous entry 859,85946 not in local log (mismatch past end of log)
2020-06-02T16:58:37Z|01570|raft|INFO|rejecting append_request because previous entry 859,85946 not in local log (mismatch past end of log)
2020-06-02T16:58:37Z|01571|raft|INFO|rejecting append_request because previous entry 859,85946 not in local log (mismatch past end of log)
2020-06-02T16:58:37Z|01572|raft|INFO|rejecting append_request because previous entry 859,85946 not in local log (mismatch past end of log)
2020-06-02T16:58:37Z|01573|raft|INFO|rejecting append_request because previous entry 859,85946 not in local log (mismatch past end of log)
2020-06-02T16:58:37Z|01574|raft|INFO|rejecting append_request because previous entry 859,85946 not in local log (mismatch past end of log)
2020-06-02T16:58:37Z|01575|raft|INFO|rejecting append_request because previous entry 859,85946 not in local log (mismatch past end of log)
2020-06-02T16:58:37Z|01576|reconnect|WARN|ssl:10.0.173.114:60982: connection dropped (Protocol error)
2020-06-02T16:58:37Z|01577|reconnect|INFO|ssl:10.0.135.159:9644: connection closed by peer
2020-06-02T16:58:38Z|01578|raft|INFO|ssl:10.0.135.159:41758: learned server ID 77d9
2020-06-02T16:58:38Z|01579|raft|INFO|ssl:10.0.135.159:41758: learned remote address ssl:10.0.135.159:9644
2020-06-02T16:58:38Z|01580|reconnect|INFO|ssl:10.0.135.159:9644: connecting...
2020-06-02T16:58:38Z|01581|reconnect|INFO|ssl:10.0.135.159:9644: connected
2020-06-02T16:58:39Z|01582|raft|INFO|ssl:10.0.173.114:32784: learned server ID db04
2020-06-02T16:58:39Z|01583|raft|INFO|ssl:10.0.173.114:32784: learned remote address ssl:10.0.173.114:9644
2020-06-02T16:59:21Z|01584|raft|INFO|server 77d9 is leader for term 860
2020-06-02T17:15:41Z|01585|timeval|WARN|Unreasonably long 7522ms poll interval (6884ms user, 169ms system)
2020-06-02T17:15:41Z|01586|timeval|WARN|faults: 93 minor, 0 major
2020-06-02T17:15:41Z|01587|timeval|WARN|disk: 0 reads, 370496 writes
2020-06-02T17:15:41Z|01588|timeval|WARN|context switches: 791 voluntary, 13 involuntary
2020-06-02T17:15:41Z|01589|coverage|INFO|Skipping details of duplicate event coverage for hash=25b513c2
2020-06-02T17:15:41Z|01590|poll_loop|INFO|Dropped 52 log messages in last 1023 seconds (most recently, 1021 seconds ago) due to excessive rate
2020-06-02T17:15:41Z|01591|poll_loop|INFO|wakeup due to [POLLIN] on fd 13 (0.0.0.0:9642<->) at lib/stream-ssl.c:963 (72% CPU usage)
2020-06-02T17:15:41Z|01592|poll_loop|INFO|wakeup due to [POLLIN] on fd 17 (10.0.221.58:41044<->10.0.135.159:9644) at lib/stream-ssl.c:827 (72% CPU usage)
2020-06-02T17:15:41Z|01593|poll_loop|INFO|wakeup due to [POLLIN] on fd 25 (10.0.221.58:46638<->10.0.173.114:9644) at lib/stream-ssl.c:827 (72% CPU usage)
2020-06-02T17:15:41Z|01594|poll_loop|INFO|wakeup due to [POLLIN] on fd 26 (10.0.221.58:9644<->10.0.173.114:32784) at lib/stream-ssl.c:827 (72% CPU usage)
2020-06-02T17:15:41Z|01595|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T17:15:41Z|01596|jsonrpc|WARN|ssl:10.0.173.114:44656: receive error: Protocol error
2020-06-02T17:15:41Z|01597|poll_loop|INFO|wakeup due to [POLLIN] on fd 59 (10.0.221.58:9642<->10.0.130.210:40372) at lib/stream-ssl.c:827 (72% CPU usage)
2020-06-02T17:15:41Z|01598|poll_loop|INFO|wakeup due to [POLLIN] on fd 37 (10.0.221.58:9642<->10.0.173.20:41130) at lib/stream-ssl.c:827 (72% CPU usage)
2020-06-02T17:15:41Z|01599|poll_loop|INFO|wakeup due to [POLLIN] on fd 47 (10.0.221.58:9642<->10.0.216.234:47704) at lib/stream-ssl.c:827 (72% CPU usage)
2020-06-02T17:15:41Z|01600|poll_loop|INFO|wakeup due to [POLLIN] on fd 66 (10.0.221.58:9642<->10.0.133.213:50188) at lib/stream-ssl.c:827 (72% CPU usage)
2020-06-02T17:15:41Z|01601|poll_loop|INFO|wakeup due to [POLLIN] on fd 58 (10.0.221.58:9642<->10.0.163.90:50668) at lib/stream-ssl.c:827 (72% CPU usage)
2020-06-02T17:15:41Z|01602|poll_loop|INFO|wakeup due to [POLLIN] on fd 80 (10.0.221.58:9642<->10.0.155.11:39120) at lib/stream-ssl.c:827 (72% CPU usage)
2020-06-02T17:15:41Z|01603|reconnect|WARN|ssl:10.0.173.114:44656: connection dropped (Protocol error)
2020-06-02T17:16:01Z|01604|reconnect|ERR|ssl:10.0.173.114:9644: no response to inactivity probe after 5 seconds, disconnecting
2020-06-02T17:16:01Z|01605|reconnect|INFO|ssl:10.0.173.114:9644: connection dropped
2020-06-02T17:16:01Z|01606|reconnect|ERR|ssl:10.0.173.114:32784: no response to inactivity probe after 5 seconds, disconnecting
2020-06-02T17:16:02Z|01607|reconnect|INFO|ssl:10.0.173.114:9644: connecting...
2020-06-02T17:16:02Z|01608|reconnect|INFO|ssl:10.0.173.114:9644: connected
2020-06-02T17:16:02Z|01609|raft|INFO|ssl:10.0.173.114:35336: learned server ID db04
2020-06-02T17:16:02Z|01610|raft|INFO|ssl:10.0.173.114:35336: learned remote address ssl:10.0.173.114:9644
2020-06-02T17:16:45Z|01611|raft|INFO|server db04 is leader for term 861
2020-06-02T17:16:51Z|01612|reconnect|ERR|ssl:10.0.135.159:9644: no response to inactivity probe after 5 seconds, disconnecting
2020-06-02T17:16:51Z|01613|reconnect|INFO|ssl:10.0.135.159:9644: connection dropped
2020-06-02T17:16:52Z|01614|reconnect|INFO|ssl:10.0.135.159:9644: connecting...
2020-06-02T17:16:52Z|01615|reconnect|INFO|ssl:10.0.135.159:9644: connected
2020-06-02T17:26:17Z|01616|raft|INFO|server 77d9 is leader for term 862
2020-06-02T17:26:22Z|01617|reconnect|ERR|ssl:10.0.173.114:9644: no response to inactivity probe after 5 seconds, disconnecting
2020-06-02T17:26:22Z|01618|reconnect|INFO|ssl:10.0.173.114:9644: connection dropped
2020-06-02T17:26:23Z|01619|reconnect|INFO|ssl:10.0.173.114:9644: connecting...
2020-06-02T17:26:24Z|01620|raft|INFO|rejecting term 861 < current term 862 received in append_request message from server db04
2020-06-02T17:26:24Z|01621|reconnect|INFO|ssl:10.0.173.114:9644: connected
2020-06-02T17:33:48Z|01622|timeval|WARN|Unreasonably long 8156ms poll interval (7520ms user, 139ms system)
2020-06-02T17:33:48Z|01623|timeval|WARN|faults: 95 minor, 0 major
2020-06-02T17:33:48Z|01624|timeval|WARN|disk: 0 reads, 380672 writes
2020-06-02T17:33:48Z|01625|timeval|WARN|context switches: 183 voluntary, 64 involuntary
2020-06-02T17:33:48Z|01626|coverage|INFO|Skipping details of duplicate event coverage for hash=25b513c2
2020-06-02T17:33:48Z|01627|poll_loop|INFO|Dropped 45 log messages in last 1087 seconds (most recently, 1084 seconds ago) due to excessive rate
2020-06-02T17:33:48Z|01628|poll_loop|INFO|wakeup due to [POLLIN] on fd 13 (0.0.0.0:9642<->) at lib/stream-ssl.c:963 (91% CPU usage)
2020-06-02T17:33:48Z|01629|poll_loop|INFO|wakeup due to [POLLIN] on fd 14 (/var/run/ovn/ovnsb_db.ctl<->) at lib/stream-fd.c:274 (91% CPU usage)
2020-06-02T17:33:48Z|01630|poll_loop|INFO|wakeup due to [POLLIN] on fd 16 (10.0.221.58:9644<->10.0.135.159:41758) at lib/stream-ssl.c:827 (91% CPU usage)
2020-06-02T17:33:48Z|01631|poll_loop|INFO|wakeup due to [POLLIN] on fd 17 (10.0.221.58:51024<->10.0.135.159:9644) at lib/stream-ssl.c:827 (91% CPU usage)
2020-06-02T17:33:48Z|01632|poll_loop|INFO|wakeup due to [POLLIN] on fd 44 (10.0.221.58:9642<->10.0.204.143:38318) at lib/stream-ssl.c:827 (91% CPU usage)
2020-06-02T17:33:48Z|01633|poll_loop|INFO|wakeup due to [POLLIN] on fd 25 (10.0.221.58:44012<->10.0.173.114:9644) at lib/stream-ssl.c:827 (91% CPU usage)
2020-06-02T17:33:48Z|01634|poll_loop|INFO|wakeup due to [POLLIN] on fd 26 (10.0.221.58:9644<->10.0.173.114:35336) at lib/stream-ssl.c:827 (91% CPU usage)
2020-06-02T17:33:48Z|01635|poll_loop|INFO|wakeup due to [POLLIN] on fd 36 (10.0.221.58:9642<->10.0.217.62:53956) at lib/stream-ssl.c:827 (91% CPU usage)
2020-06-02T17:33:48Z|01636|poll_loop|INFO|wakeup due to [POLLIN] on fd 132 (10.0.221.58:9642<->10.0.203.69:56366) at lib/stream-ssl.c:827 (91% CPU usage)
2020-06-02T17:33:48Z|01637|jsonrpc|WARN|unix#742: send error: Broken pipe
2020-06-02T17:33:48Z|01638|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T17:33:48Z|01639|jsonrpc|WARN|ssl:10.0.135.159:58268: receive error: Protocol error
2020-06-02T17:33:48Z|01640|stream_ssl|WARN|SSL_write: system error (Broken pipe)
2020-06-02T17:33:48Z|01641|jsonrpc|WARN|ssl:10.0.135.159:41758: send error: Broken pipe
2020-06-02T17:33:48Z|01642|poll_loop|INFO|wakeup due to [POLLIN] on fd 13 (0.0.0.0:9642<->) at lib/stream-ssl.c:963 (91% CPU usage)
2020-06-02T17:33:48Z|01643|reconnect|WARN|ssl:10.0.135.159:58268: connection dropped (Protocol error)
2020-06-02T17:33:48Z|01644|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T17:33:48Z|01645|jsonrpc|WARN|ssl:10.0.173.114:51528: receive error: Protocol error
2020-06-02T17:33:48Z|01646|raft|INFO|rejecting append_request because previous entry 862,87377 not in local log (mismatch past end of log)
2020-06-02T17:33:48Z|01647|raft|INFO|rejecting append_request because previous entry 862,87377 not in local log (mismatch past end of log)
2020-06-02T17:33:48Z|01648|raft|INFO|rejecting append_request because previous entry 862,87378 not in local log (mismatch past end of log)
2020-06-02T17:33:48Z|01649|raft|INFO|rejecting append_request because previous entry 862,87378 not in local log (mismatch past end of log)
2020-06-02T17:33:48Z|01650|raft|INFO|rejecting append_request because previous entry 862,87378 not in local log (mismatch past end of log)
2020-06-02T17:33:48Z|01651|reconnect|WARN|ssl:10.0.135.159:41758: connection dropped (Broken pipe)
2020-06-02T17:33:48Z|01652|raft|INFO|ssl:10.0.135.159:41244: learned server ID 77d9
2020-06-02T17:33:48Z|01653|raft|INFO|ssl:10.0.135.159:41244: learned remote address ssl:10.0.135.159:9644
2020-06-02T17:33:48Z|01654|reconnect|WARN|ssl:10.0.173.114:51528: connection dropped (Protocol error)
2020-06-02T17:36:31Z|01655|raft|INFO|term 863: 1443 ms timeout expired, starting election
2020-06-02T17:36:31Z|01656|raft|INFO|term 863: elected leader by 2+ of 3 servers
2020-06-02T17:36:38Z|01657|reconnect|ERR|ssl:10.0.135.159:9644: no response to inactivity probe after 5 seconds, disconnecting
2020-06-02T17:36:38Z|01658|reconnect|INFO|ssl:10.0.135.159:9644: connection dropped
2020-06-02T17:36:38Z|01659|raft|INFO|rejecting term 862 < current term 863 received in append_request message from server 77d9
2020-06-02T17:36:38Z|01660|raft|INFO|90d3 received "prerequisite check failed" reply from 77d9 for unknown command
2020-06-02T17:36:38Z|01661|raft|INFO|rejecting term 862 < current term 863 received in vote_reply message from server 77d9
2020-06-02T17:36:38Z|01662|raft|INFO|current entry eid a455db12-62c3-43fc-bc7f-511271d3d33b does not match prerequisite 8304fc26-b8b7-40f7-9420-2e3d5cfccc3f in execute_command_request
2020-06-02T17:36:38Z|01663|raft|INFO|current entry eid a455db12-62c3-43fc-bc7f-511271d3d33b does not match prerequisite 8304fc26-b8b7-40f7-9420-2e3d5cfccc3f in execute_command_request
2020-06-02T17:36:39Z|01664|reconnect|INFO|ssl:10.0.135.159:9644: connecting...
2020-06-02T17:36:39Z|01665|reconnect|INFO|ssl:10.0.135.159:9644: connected
2020-06-02T17:37:33Z|01666|reconnect|ERR|ssl:10.0.173.114:9644: no response to inactivity probe after 5 seconds, disconnecting
2020-06-02T17:37:33Z|01667|reconnect|INFO|ssl:10.0.173.114:9644: connection dropped
2020-06-02T17:37:33Z|01668|raft|INFO|current entry eid 835040bb-bca2-43a9-9b48-3c17a40885d2 does not match prerequisite 4347ae2f-ea55-4135-b309-459dbeb21bb6 in execute_command_request
2020-06-02T17:37:33Z|01669|raft|INFO|current entry eid 835040bb-bca2-43a9-9b48-3c17a40885d2 does not match prerequisite 4347ae2f-ea55-4135-b309-459dbeb21bb6 in execute_command_request
2020-06-02T17:37:33Z|01670|raft|INFO|current entry eid 835040bb-bca2-43a9-9b48-3c17a40885d2 does not match prerequisite 4347ae2f-ea55-4135-b309-459dbeb21bb6 in execute_command_request
2020-06-02T17:37:33Z|01671|raft|INFO|current entry eid 835040bb-bca2-43a9-9b48-3c17a40885d2 does not match prerequisite 4347ae2f-ea55-4135-b309-459dbeb21bb6 in execute_command_request
2020-06-02T17:37:33Z|01672|raft|INFO|current entry eid 835040bb-bca2-43a9-9b48-3c17a40885d2 does not match prerequisite 4347ae2f-ea55-4135-b309-459dbeb21bb6 in execute_command_request
2020-06-02T17:37:33Z|01673|raft|INFO|current entry eid 835040bb-bca2-43a9-9b48-3c17a40885d2 does not match prerequisite 4347ae2f-ea55-4135-b309-459dbeb21bb6 in execute_command_request
2020-06-02T17:37:34Z|01674|reconnect|INFO|ssl:10.0.173.114:9644: connecting...
2020-06-02T17:37:34Z|01675|reconnect|INFO|ssl:10.0.173.114:9644: connected
2020-06-02T17:39:08Z|01676|raft|INFO|Dropped 69 log messages in last 94 seconds (most recently, 94 seconds ago) due to excessive rate
2020-06-02T17:39:08Z|01677|raft|INFO|current entry eid 224edfa6-5b2b-4805-8621-ea607d740fd0 does not match prerequisite 94e83d9c-65e7-4ef8-9400-d2d79a54a6a2 in execute_command_request
2020-06-02T17:40:53Z|01678|raft|INFO|current entry eid edf8f900-521c-4945-8f8b-7301ba54ca20 does not match prerequisite d3a6f0f0-b5d8-458d-97f6-b21c235e99cc in execute_command_request
2020-06-02T17:42:23Z|01679|raft|INFO|current entry eid 713e96bf-5e5c-4e73-a461-7564fae68b37 does not match prerequisite 49f3a08f-419a-4a31-b5a8-423c96ad17e3 in execute_command_request
2020-06-02T17:49:44Z|01680|reconnect|ERR|ssl:10.0.173.114:9644: no response to inactivity probe after 5 seconds, disconnecting
2020-06-02T17:49:44Z|01681|reconnect|INFO|ssl:10.0.173.114:9644: connection dropped
2020-06-02T17:49:45Z|01682|reconnect|INFO|ssl:10.0.173.114:9644: connecting...
2020-06-02T17:49:46Z|01683|reconnect|INFO|ssl:10.0.173.114:9644: connection attempt timed out
2020-06-02T17:49:46Z|01684|reconnect|INFO|ssl:10.0.173.114:9644: waiting 2 seconds before reconnect
2020-06-02T17:49:48Z|01685|reconnect|INFO|ssl:10.0.173.114:9644: connecting...
2020-06-02T17:49:48Z|01686|reconnect|INFO|ssl:10.0.173.114:9644: connected
2020-06-02T17:50:29Z|01687|reconnect|ERR|ssl:10.0.135.159:9644: no response to inactivity probe after 5 seconds, disconnecting
2020-06-02T17:50:29Z|01688|reconnect|INFO|ssl:10.0.135.159:9644: connection dropped
2020-06-02T17:50:30Z|01689|reconnect|INFO|ssl:10.0.135.159:9644: connecting...
2020-06-02T17:50:30Z|01690|reconnect|INFO|ssl:10.0.135.159:9644: connected
2020-06-02T17:53:06Z|01691|timeval|WARN|Unreasonably long 7944ms poll interval (7181ms user, 184ms system)
2020-06-02T17:53:06Z|01692|timeval|WARN|faults: 611 minor, 0 major
2020-06-02T17:53:06Z|01693|timeval|WARN|disk: 0 reads, 393936 writes
2020-06-02T17:53:06Z|01694|timeval|WARN|context switches: 1445 voluntary, 33 involuntary
2020-06-02T17:53:06Z|01695|coverage|INFO|Skipping details of duplicate event coverage for hash=25b513c2
2020-06-02T17:53:06Z|01696|poll_loop|INFO|Dropped 87 log messages in last 1158 seconds (most recently, 1155 seconds ago) due to excessive rate
2020-06-02T17:53:06Z|01697|poll_loop|INFO|wakeup due to [POLLIN] on fd 115 (10.0.221.58:9644<->10.0.135.159:41244) at lib/stream-ssl.c:827 (85% CPU usage)
2020-06-02T17:53:06Z|01698|poll_loop|INFO|wakeup due to [POLLIN] on fd 13 (0.0.0.0:9642<->) at lib/stream-ssl.c:963 (85% CPU usage)
2020-06-02T17:53:06Z|01699|poll_loop|INFO|wakeup due to [POLLIN] on fd 14 (/var/run/ovn/ovnsb_db.ctl<->) at lib/stream-fd.c:274 (85% CPU usage)
2020-06-02T17:53:06Z|01700|poll_loop|INFO|wakeup due to [POLLIN] on fd 16 (10.0.221.58:41324<->10.0.135.159:9644) at lib/stream-ssl.c:827 (85% CPU usage)
2020-06-02T17:53:06Z|01701|poll_loop|INFO|wakeup due to [POLLIN] on fd 15 (10.0.221.58:9644<->) at lib/stream-ssl.c:963 (85% CPU usage)
2020-06-02T17:53:06Z|01702|poll_loop|INFO|wakeup due to [POLLIN] on fd 17 (10.0.221.58:9642<->10.0.173.114:57190) at lib/stream-ssl.c:827 (85% CPU usage)
2020-06-02T17:53:06Z|01703|poll_loop|INFO|wakeup due to [POLLIN] on fd 25 (10.0.221.58:56838<->10.0.173.114:9644) at lib/stream-ssl.c:827 (85% CPU usage)
2020-06-02T17:53:06Z|01704|poll_loop|INFO|wakeup due to [POLLIN] on fd 26 (10.0.221.58:9644<->10.0.173.114:35336) at lib/stream-ssl.c:827 (85% CPU usage)
2020-06-02T17:53:06Z|01705|poll_loop|INFO|wakeup due to [POLLIN] on fd 154 (10.0.221.58:9642<->10.0.221.58:55652) at lib/stream-ssl.c:827 (85% CPU usage)
2020-06-02T17:53:06Z|01706|poll_loop|INFO|wakeup due to [POLLIN] on fd 155 (10.0.221.58:9642<->10.0.135.159:32998) at lib/stream-ssl.c:827 (85% CPU usage)
2020-06-02T17:53:06Z|01707|jsonrpc|WARN|unix#858: send error: Broken pipe
2020-06-02T17:53:06Z|01708|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T17:53:06Z|01709|jsonrpc|WARN|ssl:10.0.221.58:36426: receive error: Protocol error
2020-06-02T17:53:06Z|01710|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T17:53:06Z|01711|jsonrpc|WARN|ssl:10.0.173.114:48166: receive error: Protocol error
2020-06-02T17:53:06Z|01712|reconnect|WARN|ssl:10.0.221.58:36426: connection dropped (Protocol error)
2020-06-02T17:53:06Z|01713|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T17:53:06Z|01714|jsonrpc|WARN|ssl:10.0.173.114:57956: receive error: Protocol error
2020-06-02T17:53:06Z|01715|raft|INFO|server db04 is leader for term 864
2020-06-02T17:53:06Z|01716|raft|INFO|rejecting append_request because previous entry 864,89113 not in local log (term mismatch)
2020-06-02T17:53:06Z|01717|raft|INFO|rejecting append_request because previous entry 864,89113 not in local log (term mismatch)
2020-06-02T17:53:06Z|01718|raft|INFO|rejecting append_request because previous entry 864,89113 not in local log (term mismatch)
2020-06-02T17:53:06Z|01719|raft|INFO|rejecting append_request because previous entry 864,89113 not in local log (term mismatch)
2020-06-02T17:53:06Z|01720|raft|INFO|rejecting append_request because previous entry 864,89113 not in local log (term mismatch)
2020-06-02T17:53:06Z|01721|raft|INFO|rejecting append_request because previous entry 864,89113 not in local log (term mismatch)
2020-06-02T17:53:06Z|01722|raft|INFO|rejecting append_request because previous entry 864,89113 not in local log (term mismatch)
2020-06-02T17:53:06Z|01723|raft|INFO|rejecting append_request because previous entry 864,89113 not in local log (term mismatch)
2020-06-02T17:53:06Z|01724|raft|INFO|rejecting append_request because previous entry 864,89113 not in local log (term mismatch)
2020-06-02T17:53:06Z|01725|raft|WARN|ignoring vote request received after only 0 ms (minimum election time is 1000 ms)
2020-06-02T17:53:06Z|01726|raft|INFO|90d3 truncating 1 entries from end of log
2020-06-02T17:53:06Z|01727|reconnect|WARN|ssl:10.0.173.114:48166: connection dropped (Protocol error)
2020-06-02T17:53:06Z|01728|stream_ssl|WARN|SSL_write: system error (Broken pipe)
2020-06-02T17:53:06Z|01729|jsonrpc|WARN|ssl:10.0.173.114:57190: send error: Broken pipe
2020-06-02T17:53:06Z|01730|reconnect|WARN|ssl:10.0.173.114:57956: connection dropped (Protocol error)
2020-06-02T17:53:06Z|01731|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T17:53:06Z|01732|raft|INFO|rejected append_reply (not leader)
2020-06-02T17:53:06Z|01733|raft|INFO|rejected append_reply (not leader)
2020-06-02T17:53:06Z|01734|raft|INFO|rejected append_reply (not leader)
2020-06-02T17:53:06Z|01735|raft|INFO|rejected append_reply (not leader)
2020-06-02T17:53:06Z|01736|reconnect|WARN|ssl:10.0.173.114:57190: connection dropped (Broken pipe)
2020-06-02T17:53:06Z|01737|reconnect|WARN|ssl:10.0.173.114:58158: connection dropped (Protocol error)
2020-06-02T17:53:06Z|01738|raft|INFO|ssl:10.0.135.159:58210: learned server ID 77d9
2020-06-02T17:53:06Z|01739|raft|INFO|ssl:10.0.135.159:58210: learned remote address ssl:10.0.135.159:9644
2020-06-02T17:53:07Z|01740|raft|INFO|ssl:10.0.173.114:48382: learned server ID db04
2020-06-02T17:53:07Z|01741|raft|INFO|ssl:10.0.173.114:48382: learned remote address ssl:10.0.173.114:9644
2020-06-02T18:06:05Z|01742|raft|INFO|term 865: 1635 ms timeout expired, starting election
2020-06-02T18:06:05Z|01743|raft|INFO|term 865: elected leader by 2+ of 3 servers
2020-06-02T18:06:11Z|01744|raft|INFO|rejecting term 864 < current term 865 received in vote_reply message from server db04
2020-06-02T18:06:41Z|01745|reconnect|ERR|ssl:10.0.135.159:9644: no response to inactivity probe after 5 seconds, disconnecting
2020-06-02T18:06:41Z|01746|reconnect|INFO|ssl:10.0.135.159:9644: connection dropped
2020-06-02T18:06:42Z|01747|reconnect|INFO|ssl:10.0.135.159:9644: connecting...
2020-06-02T18:06:42Z|01748|reconnect|INFO|ssl:10.0.135.159:9644: connected
2020-06-02T18:09:15Z|01749|timeval|WARN|Unreasonably long 8034ms poll interval (7310ms user, 171ms system)
2020-06-02T18:09:15Z|01750|timeval|WARN|faults: 611 minor, 0 major
2020-06-02T18:09:15Z|01751|timeval|WARN|disk: 0 reads, 393944 writes
2020-06-02T18:09:15Z|01752|timeval|WARN|context switches: 1765 voluntary, 15 involuntary
2020-06-02T18:09:15Z|01753|coverage|INFO|Skipping details of duplicate event coverage for hash=25b513c2
2020-06-02T18:09:15Z|01754|poll_loop|INFO|Dropped 56 log messages in last 970 seconds (most recently, 967 seconds ago) due to excessive rate
2020-06-02T18:09:15Z|01755|poll_loop|INFO|wakeup due to [POLLIN] on fd 13 (0.0.0.0:9642<->) at lib/stream-ssl.c:963 (71% CPU usage)
2020-06-02T18:09:15Z|01756|poll_loop|INFO|wakeup due to [POLLIN] on fd 14 (/var/run/ovn/ovnsb_db.ctl<->) at lib/stream-fd.c:274 (71% CPU usage)
2020-06-02T18:09:15Z|01757|poll_loop|INFO|wakeup due to [POLLIN] on fd 16 (10.0.221.58:50270<->10.0.135.159:9644) at lib/stream-ssl.c:827 (71% CPU usage)
2020-06-02T18:09:15Z|01758|poll_loop|INFO|wakeup due to [POLLIN] on fd 17 (10.0.221.58:9644<->10.0.135.159:58210) at lib/stream-ssl.c:827 (71% CPU usage)
2020-06-02T18:09:15Z|01759|poll_loop|INFO|wakeup due to [POLLIN] on fd 25 (10.0.221.58:56838<->10.0.173.114:9644) at lib/stream-ssl.c:827 (71% CPU usage)
2020-06-02T18:09:15Z|01760|poll_loop|INFO|wakeup due to [POLLIN] on fd 26 (10.0.221.58:9644<->10.0.173.114:48382) at lib/stream-ssl.c:827 (71% CPU usage)
2020-06-02T18:09:15Z|01761|jsonrpc|WARN|Dropped 1 log messages in last 970 seconds (most recently, 970 seconds ago) due to excessive rate
2020-06-02T18:09:15Z|01762|jsonrpc|WARN|unix#955: send error: Broken pipe
2020-06-02T18:09:15Z|01763|stream_ssl|WARN|SSL_write: system error (Broken pipe)
2020-06-02T18:09:15Z|01764|jsonrpc|WARN|ssl:10.0.221.58:43726: send error: Broken pipe
2020-06-02T18:09:15Z|01765|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T18:09:15Z|01766|jsonrpc|WARN|ssl:10.0.173.114:59934: receive error: Protocol error
2020-06-02T18:09:15Z|01767|raft|WARN|ignoring vote request received as leader
2020-06-02T18:09:15Z|01768|raft|INFO|server db04 is leader for term 866
2020-06-02T18:09:15Z|01769|poll_loop|INFO|wakeup due to [POLLIN] on fd 13 (0.0.0.0:9642<->) at lib/stream-ssl.c:963 (71% CPU usage)
2020-06-02T18:09:15Z|01770|stream_ssl|WARN|SSL_write: system error (Broken pipe)
2020-06-02T18:09:15Z|01771|jsonrpc|WARN|ssl:10.0.135.159:57454: send error: Broken pipe
2020-06-02T18:09:15Z|01772|stream_ssl|WARN|SSL_write: system error (Broken pipe)
2020-06-02T18:09:15Z|01773|jsonrpc|WARN|ssl:10.0.173.114:54914: send error: Broken pipe
2020-06-02T18:09:15Z|01774|reconnect|WARN|ssl:10.0.221.58:43726: connection dropped (Broken pipe)
2020-06-02T18:09:15Z|01775|reconnect|WARN|ssl:10.0.173.114:59934: connection dropped (Protocol error)
2020-06-02T18:09:15Z|01776|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T18:09:15Z|01777|poll_loop|INFO|wakeup due to [POLLIN] on fd 13 (0.0.0.0:9642<->) at lib/stream-ssl.c:963 (71% CPU usage)
2020-06-02T18:09:15Z|01778|reconnect|WARN|ssl:10.0.135.159:57454: connection dropped (Broken pipe)
2020-06-02T18:09:15Z|01779|reconnect|WARN|ssl:10.0.173.114:54914: connection dropped (Broken pipe)
2020-06-02T18:09:15Z|01780|reconnect|WARN|ssl:10.0.173.114:59998: connection dropped (Protocol error)
2020-06-02T18:09:15Z|01781|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T18:09:15Z|01782|poll_loop|INFO|wakeup due to 0-ms timeout at ssl:10.0.135.159:59738 (71% CPU usage)
2020-06-02T18:09:15Z|01783|reconnect|WARN|ssl:10.0.135.159:59738: connection dropped (Protocol error)
2020-06-02T18:09:16Z|01784|poll_loop|INFO|wakeup due to [POLLIN] on fd 26 (10.0.221.58:9644<->10.0.173.114:48382) at lib/stream-ssl.c:827 (71% CPU usage)
2020-06-02T18:23:11Z|01785|reconnect|ERR|ssl:10.0.135.159:9644: no response to inactivity probe after 5 seconds, disconnecting
2020-06-02T18:23:11Z|01786|reconnect|INFO|ssl:10.0.135.159:9644: connection dropped
2020-06-02T18:23:11Z|01787|reconnect|ERR|ssl:10.0.135.159:58210: no response to inactivity probe after 5 seconds, disconnecting
2020-06-02T18:23:12Z|01788|reconnect|INFO|ssl:10.0.135.159:9644: connecting...
2020-06-02T18:23:12Z|01789|reconnect|INFO|ssl:10.0.135.159:9644: connected
2020-06-02T18:23:12Z|01790|raft|INFO|ssl:10.0.135.159:53014: learned server ID 77d9
2020-06-02T18:23:12Z|01791|raft|INFO|ssl:10.0.135.159:53014: learned remote address ssl:10.0.135.159:9644
2020-06-02T18:25:17Z|01792|raft|INFO|server 77d9 is leader for term 867
2020-06-02T18:25:40Z|01793|timeval|WARN|Unreasonably long 8149ms poll interval (7436ms user, 165ms system)
2020-06-02T18:25:40Z|01794|timeval|WARN|faults: 611 minor, 0 major
2020-06-02T18:25:40Z|01795|timeval|WARN|disk: 0 reads, 393944 writes
2020-06-02T18:25:40Z|01796|timeval|WARN|context switches: 2170 voluntary, 15 involuntary
2020-06-02T18:25:40Z|01797|coverage|INFO|Skipping details of duplicate event coverage for hash=25b513c2
2020-06-02T18:25:40Z|01798|poll_loop|INFO|Dropped 23 log messages in last 984 seconds (most recently, 981 seconds ago) due to excessive rate
2020-06-02T18:25:40Z|01799|poll_loop|INFO|wakeup due to [POLLIN] on fd 14 (/var/run/ovn/ovnsb_db.ctl<->) at lib/stream-fd.c:274 (79% CPU usage)
2020-06-02T18:25:40Z|01800|poll_loop|INFO|wakeup due to [POLLIN] on fd 16 (10.0.221.58:59356<->10.0.135.159:9644) at lib/stream-ssl.c:827 (79% CPU usage)
2020-06-02T18:25:40Z|01801|poll_loop|INFO|wakeup due to [POLLIN] on fd 15 (10.0.221.58:9644<->) at lib/stream-ssl.c:963 (79% CPU usage)
2020-06-02T18:25:40Z|01802|poll_loop|INFO|wakeup due to [POLLIN] on fd 17 (10.0.221.58:9644<->10.0.135.159:53014) at lib/stream-ssl.c:827 (79% CPU usage)
2020-06-02T18:25:40Z|01803|poll_loop|INFO|wakeup due to [POLLIN] on fd 25 (10.0.221.58:56838<->10.0.173.114:9644) at lib/stream-ssl.c:827 (79% CPU usage)
2020-06-02T18:25:40Z|01804|poll_loop|INFO|wakeup due to [POLLIN] on fd 26 (10.0.221.58:9644<->10.0.173.114:48382) at lib/stream-ssl.c:827 (79% CPU usage)
2020-06-02T18:25:40Z|01805|jsonrpc|WARN|Dropped 2 log messages in last 984 seconds (most recently, 984 seconds ago) due to excessive rate
2020-06-02T18:25:40Z|01806|jsonrpc|WARN|unix#1053: send error: Broken pipe
2020-06-02T18:25:40Z|01807|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T18:25:40Z|01808|jsonrpc|WARN|ssl:10.0.135.159:54836: receive error: Protocol error
2020-06-02T18:25:40Z|01809|poll_loop|INFO|wakeup due to [POLLIN] on fd 66 (10.0.221.58:9642<->10.0.133.213:50188) at lib/stream-ssl.c:827 (79% CPU usage)
2020-06-02T18:25:40Z|01810|poll_loop|INFO|wakeup due to [POLLIN] on fd 107 (10.0.221.58:9642<->10.0.138.166:46984) at lib/stream-ssl.c:827 (79% CPU usage)
2020-06-02T18:25:40Z|01811|poll_loop|INFO|wakeup due to [POLLIN] on fd 15 (10.0.221.58:9644<->) at lib/stream-ssl.c:963 (79% CPU usage)
2020-06-02T18:25:40Z|01812|poll_loop|INFO|wakeup due to [POLLIN] on fd 118 (10.0.221.58:9642<->10.0.183.243:51546) at lib/stream-ssl.c:827 (79% CPU usage)
2020-06-02T18:25:40Z|01813|reconnect|WARN|ssl:10.0.135.159:54836: connection dropped (Protocol error)
2020-06-02T18:25:40Z|01814|reconnect|INFO|ssl:10.0.173.114:9644: connection closed by peer
2020-06-02T18:25:40Z|01815|raft|INFO|ssl:10.0.173.114:52620: learned server ID db04
2020-06-02T18:25:40Z|01816|raft|INFO|ssl:10.0.173.114:52620: learned remote address ssl:10.0.173.114:9644
2020-06-02T18:25:41Z|01817|reconnect|INFO|ssl:10.0.173.114:9644: connecting...
2020-06-02T18:25:41Z|01818|reconnect|INFO|ssl:10.0.173.114:9644: connected
2020-06-02T18:25:41Z|01819|raft|INFO|ssl:10.0.135.159:54868: learned server ID 77d9
2020-06-02T18:25:41Z|01820|raft|INFO|ssl:10.0.135.159:54868: learned remote address ssl:10.0.135.159:9644
2020-06-02T18:39:34Z|01821|raft|INFO|server db04 is leader for term 868
2020-06-02T18:41:34Z|01822|raft|INFO|server 77d9 is leader for term 869
2020-06-02T18:43:40Z|01823|timeval|WARN|Unreasonably long 7754ms poll interval (7022ms user, 180ms system)
2020-06-02T18:43:40Z|01824|timeval|WARN|faults: 611 minor, 0 major
2020-06-02T18:43:40Z|01825|timeval|WARN|disk: 0 reads, 393936 writes
2020-06-02T18:43:40Z|01826|timeval|WARN|context switches: 1520 voluntary, 15 involuntary
2020-06-02T18:43:40Z|01827|coverage|INFO|Skipping details of duplicate event coverage for hash=25b513c2
2020-06-02T18:43:40Z|01828|poll_loop|INFO|Dropped 65 log messages in last 1081 seconds (most recently, 1078 seconds ago) due to excessive rate
2020-06-02T18:43:40Z|01829|poll_loop|INFO|wakeup due to [POLLIN] on fd 13 (0.0.0.0:9642<->) at lib/stream-ssl.c:963 (75% CPU usage)
2020-06-02T18:43:40Z|01830|poll_loop|INFO|wakeup due to [POLLIN] on fd 16 (10.0.221.58:59356<->10.0.135.159:9644) at lib/stream-ssl.c:827 (75% CPU usage)
2020-06-02T18:43:40Z|01831|poll_loop|INFO|wakeup due to [POLLIN] on fd 17 (10.0.221.58:48440<->10.0.173.114:9644) at lib/stream-ssl.c:827 (75% CPU usage)
2020-06-02T18:43:40Z|01832|poll_loop|INFO|wakeup due to [POLLIN] on fd 25 (10.0.221.58:9644<->10.0.135.159:54868) at lib/stream-ssl.c:827 (75% CPU usage)
2020-06-02T18:43:40Z|01833|poll_loop|INFO|wakeup due to [POLLIN] on fd 153 (10.0.221.58:9644<->10.0.173.114:52620) at lib/stream-ssl.c:827 (75% CPU usage)
2020-06-02T18:43:40Z|01834|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T18:43:40Z|01835|jsonrpc|WARN|ssl:10.0.173.114:36878: receive error: Protocol error
2020-06-02T18:43:40Z|01836|poll_loop|INFO|wakeup due to [POLLIN] on fd 66 (10.0.221.58:9642<->10.0.133.213:50188) at lib/stream-ssl.c:827 (75% CPU usage)
2020-06-02T18:43:40Z|01837|poll_loop|INFO|wakeup due to [POLLIN] on fd 149 (10.0.221.58:9642<->10.0.187.141:58626) at lib/stream-ssl.c:827 (75% CPU usage)
2020-06-02T18:43:40Z|01838|poll_loop|INFO|wakeup due to [POLLIN] on fd 107 (10.0.221.58:9642<->10.0.138.166:46984) at lib/stream-ssl.c:827 (75% CPU usage)
2020-06-02T18:43:40Z|01839|poll_loop|INFO|wakeup due to [POLLIN] on fd 118 (10.0.221.58:9642<->10.0.183.243:51546) at lib/stream-ssl.c:827 (75% CPU usage)
2020-06-02T18:43:40Z|01840|poll_loop|INFO|wakeup due to [POLLIN] on fd 97 (10.0.221.58:9642<->10.0.243.129:51188) at lib/stream-ssl.c:827 (75% CPU usage)
2020-06-02T18:43:40Z|01841|reconnect|WARN|ssl:10.0.173.114:36878: connection dropped (Protocol error)
2020-06-02T18:43:40Z|01842|reconnect|INFO|ssl:10.0.173.114:9644: connection closed by peer
2020-06-02T18:43:41Z|01843|raft|INFO|ssl:10.0.173.114:55442: learned server ID db04
2020-06-02T18:43:41Z|01844|raft|INFO|ssl:10.0.173.114:55442: learned remote address ssl:10.0.173.114:9644
2020-06-02T18:43:41Z|01845|reconnect|INFO|ssl:10.0.173.114:9644: connecting...
2020-06-02T18:43:41Z|01846|reconnect|INFO|ssl:10.0.173.114:9644: connected
2020-06-02T18:55:56Z|01847|raft|INFO|server db04 is leader for term 870
2020-06-02T18:56:01Z|01848|reconnect|ERR|ssl:10.0.135.159:9644: no response to inactivity probe after 5 seconds, disconnecting
2020-06-02T18:56:01Z|01849|reconnect|INFO|ssl:10.0.135.159:9644: connection dropped
2020-06-02T18:56:02Z|01850|reconnect|INFO|ssl:10.0.135.159:9644: connecting...
2020-06-02T18:56:03Z|01851|raft|INFO|rejecting term 869 < current term 870 received in append_request message from server 77d9
2020-06-02T18:56:03Z|01852|reconnect|INFO|ssl:10.0.135.159:9644: connected
2020-06-02T18:57:56Z|01853|raft|INFO|term 871: 1719 ms timeout expired, starting election
2020-06-02T18:57:56Z|01854|raft|INFO|term 871: elected leader by 2+ of 3 servers
2020-06-02T18:58:02Z|01855|reconnect|ERR|ssl:10.0.173.114:9644: no response to inactivity probe after 5 seconds, disconnecting
2020-06-02T18:58:02Z|01856|reconnect|INFO|ssl:10.0.173.114:9644: connection dropped
2020-06-02T18:58:02Z|01857|raft|INFO|rejecting term 870 < current term 871 received in vote_reply message from server db04
2020-06-02T18:58:03Z|01858|reconnect|INFO|ssl:10.0.173.114:9644: connecting...
2020-06-02T18:58:03Z|01859|reconnect|INFO|ssl:10.0.173.114:9644: connected
2020-06-02T19:00:06Z|01860|timeval|WARN|Unreasonably long 7481ms poll interval (6769ms user, 167ms system)
2020-06-02T19:00:06Z|01861|timeval|WARN|faults: 615 minor, 0 major
2020-06-02T19:00:06Z|01862|timeval|WARN|disk: 0 reads, 393936 writes
2020-06-02T19:00:06Z|01863|timeval|WARN|context switches: 1516 voluntary, 14 involuntary
2020-06-02T19:00:06Z|01864|coverage|INFO|Skipping details of duplicate event coverage for hash=25b513c2
2020-06-02T19:00:06Z|01865|poll_loop|INFO|Dropped 57 log messages in last 986 seconds (most recently, 984 seconds ago) due to excessive rate
2020-06-02T19:00:06Z|01866|poll_loop|INFO|wakeup due to [POLLIN] on fd 115 (10.0.221.58:9642<->10.0.135.159:43506) at lib/stream-ssl.c:827 (68% CPU usage)
2020-06-02T19:00:06Z|01867|poll_loop|INFO|wakeup due to [POLLIN] on fd 13 (0.0.0.0:9642<->) at lib/stream-ssl.c:963 (68% CPU usage)
2020-06-02T19:00:06Z|01868|poll_loop|INFO|wakeup due to [POLLIN] on fd 14 (/var/run/ovn/ovnsb_db.ctl<->) at lib/stream-fd.c:274 (68% CPU usage)
2020-06-02T19:00:06Z|01869|poll_loop|INFO|wakeup due to [POLLIN] on fd 16 (10.0.221.58:49090<->10.0.135.159:9644) at lib/stream-ssl.c:827 (68% CPU usage)
2020-06-02T19:00:06Z|01870|poll_loop|INFO|wakeup due to [POLLIN] on fd 17 (10.0.221.58:9644<->10.0.173.114:55442) at lib/stream-ssl.c:827 (68% CPU usage)
2020-06-02T19:00:06Z|01871|poll_loop|INFO|wakeup due to [POLLIN] on fd 25 (10.0.221.58:9644<->10.0.135.159:54868) at lib/stream-ssl.c:827 (68% CPU usage)
2020-06-02T19:00:06Z|01872|poll_loop|INFO|wakeup due to [POLLIN] on fd 154 (10.0.221.58:9642<->10.0.221.58:44014) at lib/stream-ssl.c:827 (68% CPU usage)
2020-06-02T19:00:06Z|01873|poll_loop|INFO|wakeup due to [POLLIN] on fd 155 (10.0.221.58:9642<->10.0.173.114:35746) at lib/stream-ssl.c:827 (68% CPU usage)
2020-06-02T19:00:06Z|01874|poll_loop|INFO|wakeup due to [POLLIN] on fd 153 (10.0.221.58:37962<->10.0.173.114:9644) at lib/stream-ssl.c:827 (68% CPU usage)
2020-06-02T19:00:06Z|01875|jsonrpc|WARN|unix#1260: send error: Broken pipe
2020-06-02T19:00:06Z|01876|stream_ssl|WARN|SSL_accept: system error (Success)
2020-06-02T19:00:06Z|01877|jsonrpc|WARN|ssl:10.0.173.114:40590: receive error: Protocol error
2020-06-02T19:00:06Z|01878|raft|WARN|ignoring vote request received as leader
2020-06-02T19:00:06Z|01879|raft|INFO|server 77d9 is leader for term 872
2020-06-02T19:00:06Z|01880|poll_loop|INFO|wakeup due to 0-ms timeout at ovsdb/jsonrpc-server.c:599 (68% CPU usage)
2020-06-02T19:00:06Z|01881|stream_ssl|WARN|SSL_write: system error (Broken pipe)
2020-06-02T19:00:06Z|01882|jsonrpc|WARN|ssl:10.0.135.159:43506: send error: Broken pipe
2020-06-02T19:00:06Z|01883|stream_ssl|WARN|SSL_write: system error (Broken pipe)
2020-06-02T19:00:06Z|01884|jsonrpc|WARN|ssl:10.0.221.58:44014: send error: Broken pipe
2020-06-02T19:00:06Z|01885|stream_ssl|WARN|SSL_write: system error (Broken pipe)
2020-06-02T19:00:06Z|01886|jsonrpc|WARN|ssl:10.0.173.114:35746: send error: Broken pipe
2020-06-02T19:00:06Z|01887|reconnect|WARN|ssl:10.0.173.114:40590: connection dropped (Protocol error)
2020-06-02T19:00:06Z|01888|reconnect|WARN|ssl:10.0.135.159:43506: connection dropped (Broken pipe)
2020-06-02T19:00:06Z|01889|reconnect|WARN|ssl:10.0.221.58:44014: connection dropped (Broken pipe)
2020-06-02T19:00:06Z|01890|reconnect|WARN|ssl:10.0.173.114:35746: connection dropped (Broken pipe)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.