Skip to content

Instantly share code, notes, and snippets.

@mfenniak
Created May 23, 2017 14:04
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save mfenniak/d1bfa2c1c94ee9980113e70ba19ca563 to your computer and use it in GitHub Desktop.
Save mfenniak/d1bfa2c1c94ee9980113e70ba19ca563 to your computer and use it in GitHub Desktop.
4.9.27-14.31.amzn.x86_64 ena issue
May 21 12:05:37 ip-172-16-15-126 ec2net: [get_meta] Trying to get http://169.254.169.254/latest/meta-data/network/interfaces/macs/06:1e:aa:71:34:ca/local-ipv4s
May 21 12:05:37 ip-172-16-15-126 ec2net: [rewrite_aliases] Rewriting aliases of eth0
May 21 12:06:50 ip-172-16-15-126 dhclient[3314]: XMT: Solicit on eth0, interval 125920ms.
May 21 12:07:59 ip-172-16-15-126 consul[3614]: 2017/05/21 12:07:59 [INFO] consul: New leader elected: i-0f14d98ae903e7de2
May 21 12:08:56 ip-172-16-15-126 dhclient[3314]: XMT: Solicit on eth0, interval 114310ms.
May 21 12:10:51 ip-172-16-15-126 dhclient[3314]: XMT: Solicit on eth0, interval 119760ms.
May 21 12:12:51 ip-172-16-15-126 dhclient[3314]: XMT: Solicit on eth0, interval 115570ms.
May 21 12:13:36 ip-172-16-15-126 consul[3614]: 2017/05/21 12:13:36 [INFO] serf: attempting reconnect to i-0bd3ab67982f1fe55 172.16.17.249:8301
May 21 12:14:46 ip-172-16-15-126 dhclient[3314]: XMT: Solicit on eth0, interval 114980ms.
May 21 12:15:54 ip-172-16-15-126 consul[3614]: 2017/05/21 12:15:54 [INFO] memberlist: Suspect i-0cc0de42055d7454f has failed, no acks received
May 21 12:15:55 ip-172-16-15-126 consul[3614]: 2017/05/21 12:15:55 [INFO] memberlist: Suspect i-06bc61f26c1304e21 has failed, no acks received
May 21 12:15:56 ip-172-16-15-126 consul[3614]: 2017/05/21 12:15:56 [INFO] memberlist: Suspect i-03dc80424912d45e2 has failed, no acks received
May 21 12:15:57 ip-172-16-15-126 consul[3614]: 2017/05/21 12:15:57 [INFO] memberlist: Suspect i-01a218a01be0b7228 has failed, no acks received
May 21 12:15:58 ip-172-16-15-126 consul[3614]: 2017/05/21 12:15:58 [INFO] memberlist: Suspect i-08b27d38fd68fa803 has failed, no acks received
May 21 12:15:59 ip-172-16-15-126 consul[3614]: 2017/05/21 12:15:59 [INFO] memberlist: Suspect i-0668c4279c75c6276 has failed, no acks received
May 21 12:16:00 ip-172-16-15-126 kernel: [335590.005610] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 533.
May 21 12:16:00 ip-172-16-15-126 kernel: [335590.010071] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 905.
May 21 12:16:00 ip-172-16-15-126 kernel: [335590.014421] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 906.
May 21 12:16:00 ip-172-16-15-126 kernel: [335590.019248] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 416.
May 21 12:16:00 ip-172-16-15-126 kernel: [335590.024266] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 417.
May 21 12:16:00 ip-172-16-15-126 kernel: [335590.028839] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 418.
May 21 12:16:00 ip-172-16-15-126 kernel: [335590.033007] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 419.
May 21 12:16:00 ip-172-16-15-126 kernel: [335590.037131] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 420.
May 21 12:16:00 ip-172-16-15-126 kernel: [335590.041450] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 421.
May 21 12:16:00 ip-172-16-15-126 kernel: [335590.045691] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 83.
May 21 12:16:00 ip-172-16-15-126 kernel: [335590.049838] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 86.
May 21 12:16:00 ip-172-16-15-126 kernel: [335590.054012] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 87.
May 21 12:16:00 ip-172-16-15-126 kernel: [335590.058086] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 89.
May 21 12:16:00 ip-172-16-15-126 kernel: [335590.062460] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 94.
May 21 12:16:00 ip-172-16-15-126 kernel: [335590.066849] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 95.
May 21 12:16:00 ip-172-16-15-126 kernel: [335590.071118] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 98.
May 21 12:16:00 ip-172-16-15-126 kernel: [335590.075747] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 99.
May 21 12:16:00 ip-172-16-15-126 kernel: [335590.080056] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 100.
May 21 12:16:00 ip-172-16-15-126 kernel: [335590.084511] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 102.
May 21 12:16:00 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:00 [INFO] memberlist: Suspect i-0d3f1ddeac019e354 has failed, no acks received
May 21 12:16:01 ip-172-16-15-126 kernel: [335591.093646] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 21.
May 21 12:16:01 ip-172-16-15-126 kernel: [335591.097647] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 92.
May 21 12:16:01 ip-172-16-15-126 kernel: [335591.101552] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 96.
May 21 12:16:01 ip-172-16-15-126 kernel: [335591.105449] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 237.
May 21 12:16:01 ip-172-16-15-126 kernel: [335591.109379] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 238.
May 21 12:16:01 ip-172-16-15-126 kernel: [335591.126336] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 244.
May 21 12:16:01 ip-172-16-15-126 kernel: [335591.130579] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 245.
May 21 12:16:01 ip-172-16-15-126 kernel: [335591.134763] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 246.
May 21 12:16:01 ip-172-16-15-126 kernel: [335591.139039] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 247.
May 21 12:16:01 ip-172-16-15-126 kernel: [335591.147302] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 249.
May 21 12:16:01 ip-172-16-15-126 kernel: [335591.151355] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 250.
May 21 12:16:01 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:01 [INFO] memberlist: Suspect i-0f6372101a741e520 has failed, no acks received
May 21 12:16:02 ip-172-16-15-126 kernel: [335592.181701] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 0, index 993.
May 21 12:16:02 ip-172-16-15-126 kernel: [335592.186420] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 0, index 994.
May 21 12:16:02 ip-172-16-15-126 kernel: [335592.190616] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 0, index 995.
May 21 12:16:02 ip-172-16-15-126 kernel: [335592.194761] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 0, index 996.
May 21 12:16:02 ip-172-16-15-126 kernel: [335592.199496] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 0, index 997.
May 21 12:16:02 ip-172-16-15-126 kernel: [335592.204210] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 0, index 1000.
May 21 12:16:02 ip-172-16-15-126 kernel: [335592.208920] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 809.
May 21 12:16:02 ip-172-16-15-126 kernel: [335592.213519] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 810.
May 21 12:16:02 ip-172-16-15-126 kernel: [335592.218263] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 811.
May 21 12:16:02 ip-172-16-15-126 kernel: [335592.222886] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 812.
May 21 12:16:02 ip-172-16-15-126 kernel: [335592.227431] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 813.
May 21 12:16:02 ip-172-16-15-126 kernel: [335592.231836] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 815.
May 21 12:16:02 ip-172-16-15-126 kernel: [335592.236406] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 171.
May 21 12:16:02 ip-172-16-15-126 kernel: [335592.241056] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 172.
May 21 12:16:02 ip-172-16-15-126 kernel: [335592.245571] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 173.
May 21 12:16:02 ip-172-16-15-126 kernel: [335592.249998] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 174.
May 21 12:16:02 ip-172-16-15-126 kernel: [335592.254277] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 175.
May 21 12:16:02 ip-172-16-15-126 kernel: [335592.258414] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 176.
May 21 12:16:02 ip-172-16-15-126 kernel: [335592.262361] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 177.
May 21 12:16:02 ip-172-16-15-126 kernel: [335592.266272] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 178.
May 21 12:16:02 ip-172-16-15-126 kernel: [335592.270505] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 525.
May 21 12:16:02 ip-172-16-15-126 kernel: [335592.274667] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 526.
May 21 12:16:02 ip-172-16-15-126 kernel: [335592.278761] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 531.
May 21 12:16:02 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:02 [INFO] memberlist: Suspect i-08e477c727ca9b536 has failed, no acks received
May 21 12:16:03 ip-172-16-15-126 kernel: [335593.301733] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 532.
May 21 12:16:03 ip-172-16-15-126 kernel: [335593.305903] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 535.
May 21 12:16:03 ip-172-16-15-126 kernel: [335593.310489] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 536.
May 21 12:16:03 ip-172-16-15-126 kernel: [335593.315117] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 907.
May 21 12:16:03 ip-172-16-15-126 kernel: [335593.319414] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 908.
May 21 12:16:03 ip-172-16-15-126 kernel: [335593.323696] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 909.
May 21 12:16:03 ip-172-16-15-126 kernel: [335593.327831] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 910.
May 21 12:16:03 ip-172-16-15-126 kernel: [335593.332092] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 911.
May 21 12:16:03 ip-172-16-15-126 kernel: [335593.336569] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 422.
May 21 12:16:03 ip-172-16-15-126 kernel: [335593.340603] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 423.
May 21 12:16:03 ip-172-16-15-126 kernel: [335593.344782] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 424.
May 21 12:16:03 ip-172-16-15-126 kernel: [335593.349244] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 425.
May 21 12:16:03 ip-172-16-15-126 kernel: [335593.353324] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 426.
May 21 12:16:03 ip-172-16-15-126 kernel: [335593.357547] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 427.
May 21 12:16:03 ip-172-16-15-126 kernel: [335593.361792] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 428.
May 21 12:16:03 ip-172-16-15-126 kernel: [335593.366003] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 429.
May 21 12:16:03 ip-172-16-15-126 kernel: [335593.370183] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 430.
May 21 12:16:03 ip-172-16-15-126 kernel: [335593.374445] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 431.
May 21 12:16:03 ip-172-16-15-126 kernel: [335593.378399] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 97.
May 21 12:16:03 ip-172-16-15-126 kernel: [335593.382585] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 110.
May 21 12:16:03 ip-172-16-15-126 kernel: [335593.387010] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 113.
May 21 12:16:03 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:03 [INFO] memberlist: Suspect i-0803418647953526e has failed, no acks received
May 21 12:16:04 ip-172-16-15-126 kernel: [335594.421767] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 104.
May 21 12:16:04 ip-172-16-15-126 kernel: [335594.426092] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 112.
May 21 12:16:04 ip-172-16-15-126 kernel: [335594.430411] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 115.
May 21 12:16:04 ip-172-16-15-126 kernel: [335594.434730] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 251.
May 21 12:16:04 ip-172-16-15-126 kernel: [335594.439046] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 252.
May 21 12:16:04 ip-172-16-15-126 kernel: [335594.443171] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 253.
May 21 12:16:04 ip-172-16-15-126 kernel: [335594.447194] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 255.
May 21 12:16:04 ip-172-16-15-126 kernel: [335594.451137] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 256.
May 21 12:16:04 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:04 [INFO] memberlist: Marking i-0cc0de42055d7454f as failed, suspect timeout reached
May 21 12:16:04 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:04 [INFO] serf: EventMemberFailed: i-0cc0de42055d7454f 172.16.16.88
May 21 12:16:04 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:04 [INFO] memberlist: Suspect i-0a0a5bfcdf25cf714 has failed, no acks received
May 21 12:16:05 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:05 [ERR] yamux: keepalive failed: i/o deadline reached
May 21 12:16:05 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:05 [ERR] agent: failed to sync remote state: rpc error: EOF
May 21 12:16:05 ip-172-16-15-126 kernel: [335595.477819] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 0, index 1001.
May 21 12:16:05 ip-172-16-15-126 kernel: [335595.482124] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 0, index 1002.
May 21 12:16:05 ip-172-16-15-126 kernel: [335595.486288] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 0, index 1003.
May 21 12:16:05 ip-172-16-15-126 kernel: [335595.490459] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 0, index 1004.
May 21 12:16:05 ip-172-16-15-126 kernel: [335595.494432] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 814.
May 21 12:16:05 ip-172-16-15-126 kernel: [335595.498505] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 816.
May 21 12:16:05 ip-172-16-15-126 kernel: [335595.503028] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 817.
May 21 12:16:05 ip-172-16-15-126 kernel: [335595.507216] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 818.
May 21 12:16:05 ip-172-16-15-126 kernel: [335595.511233] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 819.
May 21 12:16:05 ip-172-16-15-126 kernel: [335595.515394] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 179.
May 21 12:16:05 ip-172-16-15-126 kernel: [335595.519865] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 180.
May 21 12:16:05 ip-172-16-15-126 kernel: [335595.524532] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 181.
May 21 12:16:05 ip-172-16-15-126 kernel: [335595.528870] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 182.
May 21 12:16:05 ip-172-16-15-126 kernel: [335595.533590] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 183.
May 21 12:16:05 ip-172-16-15-126 kernel: [335595.538120] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 184.
May 21 12:16:05 ip-172-16-15-126 kernel: [335595.542559] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 185.
May 21 12:16:05 ip-172-16-15-126 kernel: [335595.547110] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 186.
May 21 12:16:05 ip-172-16-15-126 kernel: [335595.551720] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 537.
May 21 12:16:05 ip-172-16-15-126 kernel: [335595.555983] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 538.
May 21 12:16:05 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:05 [INFO] memberlist: Marking i-06bc61f26c1304e21 as failed, suspect timeout reached
May 21 12:16:05 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:05 [INFO] serf: EventMemberFailed: i-06bc61f26c1304e21 172.16.14.71
May 21 12:16:05 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:05 [INFO] memberlist: Suspect i-0ee769a39584ed7d7 has failed, no acks received
May 21 12:16:06 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:06 [INFO] memberlist: Marking i-03dc80424912d45e2 as failed, suspect timeout reached
May 21 12:16:06 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:06 [INFO] serf: EventMemberFailed: i-03dc80424912d45e2 172.16.14.41
May 21 12:16:06 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:06 [INFO] memberlist: Suspect i-040d2e16273351cff has failed, no acks received
May 21 12:16:06 ip-172-16-15-126 kernel: [335596.565866] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 539.
May 21 12:16:06 ip-172-16-15-126 kernel: [335596.570171] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 912.
May 21 12:16:06 ip-172-16-15-126 kernel: [335596.574194] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 913.
May 21 12:16:06 ip-172-16-15-126 kernel: [335596.578084] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 914.
May 21 12:16:06 ip-172-16-15-126 kernel: [335596.582171] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 915.
May 21 12:16:06 ip-172-16-15-126 kernel: [335596.586198] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 916.
May 21 12:16:06 ip-172-16-15-126 kernel: [335596.590397] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 917.
May 21 12:16:06 ip-172-16-15-126 kernel: [335596.594609] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 918.
May 21 12:16:06 ip-172-16-15-126 kernel: [335596.598782] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 919.
May 21 12:16:06 ip-172-16-15-126 kernel: [335596.603094] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 432.
May 21 12:16:06 ip-172-16-15-126 kernel: [335596.607003] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 433.
May 21 12:16:06 ip-172-16-15-126 kernel: [335596.610941] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 434.
May 21 12:16:06 ip-172-16-15-126 kernel: [335596.628329] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 435.
May 21 12:16:06 ip-172-16-15-126 kernel: [335596.632294] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 93.
May 21 12:16:06 ip-172-16-15-126 kernel: [335596.636191] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 101.
May 21 12:16:06 ip-172-16-15-126 kernel: [335596.640116] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 103.
May 21 12:16:06 ip-172-16-15-126 kernel: [335596.644036] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 105.
May 21 12:16:06 ip-172-16-15-126 kernel: [335596.647949] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 106.
May 21 12:16:06 ip-172-16-15-126 kernel: [335596.651876] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 107.
May 21 12:16:06 ip-172-16-15-126 kernel: [335596.655777] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 109.
May 21 12:16:06 ip-172-16-15-126 kernel: [335596.659686] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 111.
May 21 12:16:06 ip-172-16-15-126 kernel: [335596.663608] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 114.
May 21 12:16:06 ip-172-16-15-126 kernel: [335596.667519] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 116.
May 21 12:16:06 ip-172-16-15-126 kernel: [335596.671506] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 117.
May 21 12:16:06 ip-172-16-15-126 kernel: [335596.675415] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 118.
May 21 12:16:06 ip-172-16-15-126 kernel: [335596.679329] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 119.
May 21 12:16:06 ip-172-16-15-126 kernel: [335596.683233] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 120.
May 21 12:16:06 ip-172-16-15-126 kernel: [335596.687148] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 121.
May 21 12:16:06 ip-172-16-15-126 kernel: [335596.691075] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 122.
May 21 12:16:07 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:07 [ERR] agent: coordinate update error: rpc error: failed to get conn: dial tcp 172.16.14.67:8300: getsockopt: no route to host
May 21 12:16:07 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:07 [INFO] memberlist: Marking i-01a218a01be0b7228 as failed, suspect timeout reached
May 21 12:16:07 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:07 [INFO] serf: EventMemberFailed: i-01a218a01be0b7228 172.16.16.114
May 21 12:16:07 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:07 [INFO] memberlist: Suspect i-0f0f87b95a476583d has failed, no acks received
May 21 12:16:07 ip-172-16-15-126 kernel: [335597.721917] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 108.
May 21 12:16:07 ip-172-16-15-126 kernel: [335597.725956] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 123.
May 21 12:16:07 ip-172-16-15-126 kernel: [335597.729955] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 125.
May 21 12:16:07 ip-172-16-15-126 kernel: [335597.733997] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 127.
May 21 12:16:07 ip-172-16-15-126 kernel: [335597.737989] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 128.
May 21 12:16:07 ip-172-16-15-126 kernel: [335597.741943] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 129.
May 21 12:16:07 ip-172-16-15-126 kernel: [335597.745902] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 257.
May 21 12:16:07 ip-172-16-15-126 kernel: [335597.749889] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 258.
May 21 12:16:07 ip-172-16-15-126 kernel: [335597.753801] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 259.
May 21 12:16:07 ip-172-16-15-126 kernel: [335597.757649] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 260.
May 21 12:16:08 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:08 [INFO] memberlist: Marking i-08b27d38fd68fa803 as failed, suspect timeout reached
May 21 12:16:08 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:08 [INFO] serf: EventMemberFailed: i-08b27d38fd68fa803 172.16.16.243
May 21 12:16:08 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:08 [INFO] memberlist: Suspect i-0de2d29024b43cb6d has failed, no acks received
May 21 12:16:08 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:08 [INFO] consul: removing server i-08b27d38fd68fa803 (Addr: 172.16.16.243:8300) (DC: dc1)
May 21 12:16:09 ip-172-16-15-126 kernel: [335598.773947] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 0, index 1005.
May 21 12:16:09 ip-172-16-15-126 kernel: [335598.778595] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 0, index 1008.
May 21 12:16:09 ip-172-16-15-126 kernel: [335598.782871] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 820.
May 21 12:16:09 ip-172-16-15-126 kernel: [335598.787066] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 821.
May 21 12:16:09 ip-172-16-15-126 kernel: [335598.791196] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 823.
May 21 12:16:09 ip-172-16-15-126 kernel: [335598.795361] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 824.
May 21 12:16:09 ip-172-16-15-126 kernel: [335598.799631] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 825.
May 21 12:16:09 ip-172-16-15-126 kernel: [335598.803782] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 826.
May 21 12:16:09 ip-172-16-15-126 kernel: [335598.807906] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 827.
May 21 12:16:09 ip-172-16-15-126 kernel: [335598.812074] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 828.
May 21 12:16:09 ip-172-16-15-126 kernel: [335598.816276] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 187.
May 21 12:16:09 ip-172-16-15-126 kernel: [335598.820616] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 188.
May 21 12:16:09 ip-172-16-15-126 kernel: [335598.824935] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 189.
May 21 12:16:09 ip-172-16-15-126 kernel: [335598.829714] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 190.
May 21 12:16:09 ip-172-16-15-126 kernel: [335598.834064] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 191.
May 21 12:16:09 ip-172-16-15-126 kernel: [335598.838281] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 540.
May 21 12:16:09 ip-172-16-15-126 kernel: [335598.842491] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 541.
May 21 12:16:09 ip-172-16-15-126 kernel: [335598.846633] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 542.
May 21 12:16:09 ip-172-16-15-126 kernel: [335598.850898] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 543.
May 21 12:16:09 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:09 [INFO] memberlist: Marking i-0668c4279c75c6276 as failed, suspect timeout reached
May 21 12:16:09 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:09 [INFO] serf: EventMemberFailed: i-0668c4279c75c6276 172.16.16.47
May 21 12:16:09 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:09 [INFO] memberlist: Suspect i-09013c5adb8f3f726 has failed, no acks received
May 21 12:16:10 ip-172-16-15-126 kernel: [335599.861980] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 544.
May 21 12:16:10 ip-172-16-15-126 kernel: [335599.866109] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 546.
May 21 12:16:10 ip-172-16-15-126 kernel: [335599.870014] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 547.
May 21 12:16:10 ip-172-16-15-126 kernel: [335599.873937] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 920.
May 21 12:16:10 ip-172-16-15-126 kernel: [335599.877856] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 921.
May 21 12:16:10 ip-172-16-15-126 kernel: [335599.881790] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 922.
May 21 12:16:10 ip-172-16-15-126 kernel: [335599.885673] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 923.
May 21 12:16:10 ip-172-16-15-126 kernel: [335599.889606] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 436.
May 21 12:16:10 ip-172-16-15-126 kernel: [335599.893514] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 437.
May 21 12:16:10 ip-172-16-15-126 kernel: [335599.897422] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 438.
May 21 12:16:10 ip-172-16-15-126 kernel: [335599.901323] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 439.
May 21 12:16:10 ip-172-16-15-126 kernel: [335599.905224] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 440.
May 21 12:16:10 ip-172-16-15-126 kernel: [335599.909145] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 441.
May 21 12:16:10 ip-172-16-15-126 kernel: [335599.913088] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 91.
May 21 12:16:10 ip-172-16-15-126 kernel: [335599.916959] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 124.
May 21 12:16:10 ip-172-16-15-126 kernel: [335599.920873] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 126.
May 21 12:16:10 ip-172-16-15-126 kernel: [335599.924787] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 130.
May 21 12:16:10 ip-172-16-15-126 kernel: [335599.928700] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 131.
May 21 12:16:10 ip-172-16-15-126 kernel: [335599.932622] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 132.
May 21 12:16:10 ip-172-16-15-126 kernel: [335599.936531] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 133.
May 21 12:16:10 ip-172-16-15-126 kernel: [335599.940467] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 134.
May 21 12:16:10 ip-172-16-15-126 kernel: [335599.944394] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 135.
May 21 12:16:10 ip-172-16-15-126 kernel: [335599.948308] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 136.
May 21 12:16:10 ip-172-16-15-126 kernel: [335599.952280] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 139.
May 21 12:16:10 ip-172-16-15-126 kernel: [335599.956239] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 144.
May 21 12:16:10 ip-172-16-15-126 kernel: [335599.960202] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 146.
May 21 12:16:10 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:10 [INFO] memberlist: Marking i-0d3f1ddeac019e354 as failed, suspect timeout reached
May 21 12:16:10 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:10 [INFO] serf: EventMemberFailed: i-0d3f1ddeac019e354 172.16.17.176
May 21 12:16:10 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:10 [INFO] memberlist: Suspect i-0552ac87d19ad1bad has failed, no acks received
May 21 12:16:11 ip-172-16-15-126 kernel: [335600.982034] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 138.
May 21 12:16:11 ip-172-16-15-126 kernel: [335600.986592] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 140.
May 21 12:16:11 ip-172-16-15-126 kernel: [335600.990966] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 141.
May 21 12:16:11 ip-172-16-15-126 kernel: [335600.995277] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 143.
May 21 12:16:11 ip-172-16-15-126 kernel: [335600.999570] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 145.
May 21 12:16:11 ip-172-16-15-126 kernel: [335601.003865] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 148.
May 21 12:16:11 ip-172-16-15-126 kernel: [335601.008139] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 149.
May 21 12:16:11 ip-172-16-15-126 kernel: [335601.012055] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 150.
May 21 12:16:11 ip-172-16-15-126 kernel: [335601.015886] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 254.
May 21 12:16:11 ip-172-16-15-126 kernel: [335601.019808] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 261.
May 21 12:16:11 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:11 [INFO] memberlist: Marking i-0f6372101a741e520 as failed, suspect timeout reached
May 21 12:16:11 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:11 [INFO] serf: EventMemberFailed: i-0f6372101a741e520 172.16.14.160
May 21 12:16:11 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:11 [INFO] memberlist: Suspect i-06675422c58ce18ce has failed, no acks received
May 21 12:16:12 ip-172-16-15-126 kernel: [335602.038051] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 0, index 1009.
May 21 12:16:12 ip-172-16-15-126 kernel: [335602.042121] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 822.
May 21 12:16:12 ip-172-16-15-126 kernel: [335602.046064] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 829.
May 21 12:16:12 ip-172-16-15-126 kernel: [335602.050036] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 830.
May 21 12:16:12 ip-172-16-15-126 kernel: [335602.053996] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 831.
May 21 12:16:12 ip-172-16-15-126 kernel: [335602.057926] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 832.
May 21 12:16:12 ip-172-16-15-126 kernel: [335602.061876] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 833.
May 21 12:16:12 ip-172-16-15-126 kernel: [335602.065850] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 192.
May 21 12:16:12 ip-172-16-15-126 kernel: [335602.082613] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 193.
May 21 12:16:12 ip-172-16-15-126 kernel: [335602.086549] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 194.
May 21 12:16:12 ip-172-16-15-126 kernel: [335602.090484] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 195.
May 21 12:16:12 ip-172-16-15-126 kernel: [335602.094422] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 196.
May 21 12:16:12 ip-172-16-15-126 kernel: [335602.098372] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 197.
May 21 12:16:12 ip-172-16-15-126 kernel: [335602.102305] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 198.
May 21 12:16:12 ip-172-16-15-126 kernel: [335602.106209] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 199.
May 21 12:16:12 ip-172-16-15-126 kernel: [335602.110129] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 201.
May 21 12:16:12 ip-172-16-15-126 kernel: [335602.114052] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 202.
May 21 12:16:12 ip-172-16-15-126 kernel: [335602.118006] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 545.
May 21 12:16:12 ip-172-16-15-126 kernel: [335602.122035] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 548.
May 21 12:16:12 ip-172-16-15-126 kernel: [335602.126067] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 549.
May 21 12:16:12 ip-172-16-15-126 kernel: [335602.130059] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 550.
May 21 12:16:12 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:12 [INFO] memberlist: Marking i-08e477c727ca9b536 as failed, suspect timeout reached
May 21 12:16:12 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:12 [INFO] serf: EventMemberFailed: i-08e477c727ca9b536 172.16.17.83
May 21 12:16:12 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:12 [INFO] memberlist: Suspect i-0280f96a4564c8ccd has failed, no acks received
May 21 12:16:13 ip-172-16-15-126 kernel: [335603.158099] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 552.
May 21 12:16:13 ip-172-16-15-126 kernel: [335603.162157] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 924.
May 21 12:16:13 ip-172-16-15-126 kernel: [335603.166143] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 925.
May 21 12:16:13 ip-172-16-15-126 kernel: [335603.170090] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 926.
May 21 12:16:13 ip-172-16-15-126 kernel: [335603.173990] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 927.
May 21 12:16:13 ip-172-16-15-126 kernel: [335603.177906] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 928.
May 21 12:16:13 ip-172-16-15-126 kernel: [335603.181828] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 929.
May 21 12:16:13 ip-172-16-15-126 kernel: [335603.185730] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 932.
May 21 12:16:13 ip-172-16-15-126 kernel: [335603.189657] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 442.
May 21 12:16:13 ip-172-16-15-126 kernel: [335603.193560] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 443.
May 21 12:16:13 ip-172-16-15-126 kernel: [335603.197454] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 444.
May 21 12:16:13 ip-172-16-15-126 kernel: [335603.201354] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 445.
May 21 12:16:13 ip-172-16-15-126 kernel: [335603.205246] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 142.
May 21 12:16:13 ip-172-16-15-126 kernel: [335603.209141] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 147.
May 21 12:16:13 ip-172-16-15-126 kernel: [335603.213037] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 152.
May 21 12:16:13 ip-172-16-15-126 kernel: [335603.216943] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 153.
May 21 12:16:13 ip-172-16-15-126 kernel: [335603.220847] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 154.
May 21 12:16:13 ip-172-16-15-126 kernel: [335603.224782] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 155.
May 21 12:16:13 ip-172-16-15-126 kernel: [335603.228685] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 156.
May 21 12:16:13 ip-172-16-15-126 kernel: [335603.232587] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 157.
May 21 12:16:13 ip-172-16-15-126 kernel: [335603.236474] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 158.
May 21 12:16:13 ip-172-16-15-126 kernel: [335603.240378] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 159.
May 21 12:16:13 ip-172-16-15-126 kernel: [335603.244261] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 160.
May 21 12:16:13 ip-172-16-15-126 kernel: [335603.248162] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 161.
May 21 12:16:13 ip-172-16-15-126 kernel: [335603.252090] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 165.
May 21 12:16:13 ip-172-16-15-126 kernel: [335603.255992] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 168.
May 21 12:16:13 ip-172-16-15-126 kernel: [335603.259904] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 170.
May 21 12:16:13 ip-172-16-15-126 kernel: [335603.263837] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 171.
May 21 12:16:13 ip-172-16-15-126 kernel: [335603.267754] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 172.
May 21 12:16:13 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:13 [INFO] memberlist: Marking i-0803418647953526e as failed, suspect timeout reached
May 21 12:16:13 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:13 [INFO] serf: EventMemberFailed: i-0803418647953526e 172.16.17.61
May 21 12:16:13 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:13 [INFO] memberlist: Suspect i-03c4d88e2854d047b has failed, no acks received
May 21 12:16:14 ip-172-16-15-126 kernel: [335604.278157] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 162.
May 21 12:16:14 ip-172-16-15-126 kernel: [335604.282180] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 163.
May 21 12:16:14 ip-172-16-15-126 kernel: [335604.285973] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 169.
May 21 12:16:14 ip-172-16-15-126 kernel: [335604.289771] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 173.
May 21 12:16:14 ip-172-16-15-126 kernel: [335604.293592] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 262.
May 21 12:16:14 ip-172-16-15-126 kernel: [335604.297369] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 263.
May 21 12:16:14 ip-172-16-15-126 kernel: [335604.301155] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 264.
May 21 12:16:14 ip-172-16-15-126 kernel: [335604.304923] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 265.
May 21 12:16:14 ip-172-16-15-126 kernel: [335604.308688] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 266.
May 21 12:16:14 ip-172-16-15-126 kernel: [335604.312457] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 267.
May 21 12:16:14 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:14 [INFO] memberlist: Marking i-0a0a5bfcdf25cf714 as failed, suspect timeout reached
May 21 12:16:14 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:14 [INFO] serf: EventMemberFailed: i-0a0a5bfcdf25cf714 172.16.14.141
May 21 12:16:14 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:14 [INFO] memberlist: Suspect i-0af21f21158ce7c92 has failed, no acks received
May 21 12:16:15 ip-172-16-15-126 kernel: [335605.334181] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 0, index 1006.
May 21 12:16:15 ip-172-16-15-126 kernel: [335605.338200] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 0, index 1010.
May 21 12:16:15 ip-172-16-15-126 kernel: [335605.342124] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 0, index 1011.
May 21 12:16:15 ip-172-16-15-126 kernel: [335605.346053] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 834.
May 21 12:16:15 ip-172-16-15-126 kernel: [335605.349942] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 835.
May 21 12:16:15 ip-172-16-15-126 kernel: [335605.353824] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 836.
May 21 12:16:15 ip-172-16-15-126 kernel: [335605.357717] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 837.
May 21 12:16:15 ip-172-16-15-126 kernel: [335605.361651] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 838.
May 21 12:16:15 ip-172-16-15-126 kernel: [335605.365539] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 839.
May 21 12:16:15 ip-172-16-15-126 kernel: [335605.369402] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 840.
May 21 12:16:15 ip-172-16-15-126 kernel: [335605.373280] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 200.
May 21 12:16:15 ip-172-16-15-126 kernel: [335605.377153] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 203.
May 21 12:16:15 ip-172-16-15-126 kernel: [335605.381036] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 204.
May 21 12:16:15 ip-172-16-15-126 kernel: [335605.384922] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 205.
May 21 12:16:15 ip-172-16-15-126 kernel: [335605.388803] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 206.
May 21 12:16:15 ip-172-16-15-126 kernel: [335605.392706] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 207.
May 21 12:16:15 ip-172-16-15-126 kernel: [335605.396588] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 208.
May 21 12:16:15 ip-172-16-15-126 kernel: [335605.400490] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 209.
May 21 12:16:15 ip-172-16-15-126 kernel: [335605.404393] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 553.
May 21 12:16:15 ip-172-16-15-126 kernel: [335605.408259] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 554.
May 21 12:16:15 ip-172-16-15-126 kernel: [335605.412169] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 555.
May 21 12:16:15 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:15 [INFO] memberlist: Marking i-0ee769a39584ed7d7 as failed, suspect timeout reached
May 21 12:16:15 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:15 [INFO] serf: EventMemberFailed: i-0ee769a39584ed7d7 172.16.14.199
May 21 12:16:15 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:15 [INFO] memberlist: Suspect i-0f14d98ae903e7de2 has failed, no acks received
May 21 12:16:16 ip-172-16-15-126 kernel: [335606.422253] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 933.
May 21 12:16:16 ip-172-16-15-126 kernel: [335606.426747] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 934.
May 21 12:16:16 ip-172-16-15-126 kernel: [335606.431097] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 935.
May 21 12:16:16 ip-172-16-15-126 kernel: [335606.435456] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 446.
May 21 12:16:16 ip-172-16-15-126 kernel: [335606.439700] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 447.
May 21 12:16:16 ip-172-16-15-126 kernel: [335606.443966] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 448.
May 21 12:16:16 ip-172-16-15-126 kernel: [335606.448352] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 449.
May 21 12:16:16 ip-172-16-15-126 kernel: [335606.452577] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 450.
May 21 12:16:16 ip-172-16-15-126 kernel: [335606.456734] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 137.
May 21 12:16:16 ip-172-16-15-126 kernel: [335606.461016] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 151.
May 21 12:16:16 ip-172-16-15-126 kernel: [335606.465300] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 164.
May 21 12:16:16 ip-172-16-15-126 kernel: [335606.469591] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 166.
May 21 12:16:16 ip-172-16-15-126 kernel: [335606.473890] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 167.
May 21 12:16:16 ip-172-16-15-126 kernel: [335606.478065] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 176.
May 21 12:16:16 ip-172-16-15-126 kernel: [335606.482228] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 177.
May 21 12:16:16 ip-172-16-15-126 kernel: [335606.486393] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 178.
May 21 12:16:16 ip-172-16-15-126 kernel: [335606.490539] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 179.
May 21 12:16:16 ip-172-16-15-126 kernel: [335606.507563] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 181.
May 21 12:16:16 ip-172-16-15-126 kernel: [335606.511724] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 182.
May 21 12:16:16 ip-172-16-15-126 kernel: [335606.515860] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 185.
May 21 12:16:16 ip-172-16-15-126 kernel: [335606.520151] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 188.
May 21 12:16:16 ip-172-16-15-126 kernel: [335606.524560] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 191.
May 21 12:16:16 ip-172-16-15-126 kernel: [335606.529148] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 192.
May 21 12:16:16 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:16 [INFO] memberlist: Marking i-040d2e16273351cff as failed, suspect timeout reached
May 21 12:16:16 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:16 [INFO] serf: EventMemberFailed: i-040d2e16273351cff 172.16.17.130
May 21 12:16:16 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:16 [INFO] memberlist: Suspect i-02b2968ab933cf778 has failed, no acks received
May 21 12:16:16 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:16 [INFO] serf: attempting reconnect to i-040d2e16273351cff 172.16.17.130:8301
May 21 12:16:17 ip-172-16-15-126 kernel: [335607.542273] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 174.
May 21 12:16:17 ip-172-16-15-126 kernel: [335607.546530] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 183.
May 21 12:16:17 ip-172-16-15-126 kernel: [335607.550760] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 194.
May 21 12:16:17 ip-172-16-15-126 kernel: [335607.554962] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 201.
May 21 12:16:17 ip-172-16-15-126 kernel: [335607.559094] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 268.
May 21 12:16:17 ip-172-16-15-126 kernel: [335607.563218] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 269.
May 21 12:16:17 ip-172-16-15-126 kernel: [335607.567367] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 270.
May 21 12:16:17 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:17 [INFO] memberlist: Marking i-0f0f87b95a476583d as failed, suspect timeout reached
May 21 12:16:17 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:17 [INFO] serf: EventMemberFailed: i-0f0f87b95a476583d 172.16.14.143
May 21 12:16:17 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:17 [INFO] memberlist: Suspect i-055abe2ce793ae2c8 has failed, no acks received
May 21 12:16:18 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:18 [ERR] agent: failed to sync remote state: rpc error: failed to get conn: dial tcp 172.16.16.243:8300: i/o timeout
May 21 12:16:18 ip-172-16-15-126 kernel: [335608.598324] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 0, index 1007.
May 21 12:16:18 ip-172-16-15-126 kernel: [335608.602886] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 0, index 1013.
May 21 12:16:18 ip-172-16-15-126 kernel: [335608.607268] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 841.
May 21 12:16:18 ip-172-16-15-126 kernel: [335608.611594] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 842.
May 21 12:16:18 ip-172-16-15-126 kernel: [335608.615959] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 843.
May 21 12:16:18 ip-172-16-15-126 kernel: [335608.620320] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 844.
May 21 12:16:18 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:18 [INFO] memberlist: Marking i-0de2d29024b43cb6d as failed, suspect timeout reached
May 21 12:16:18 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:18 [INFO] serf: EventMemberFailed: i-0de2d29024b43cb6d 172.16.14.47
May 21 12:16:18 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:18 [INFO] memberlist: Suspect i-0757cbd488bd58eb4 has failed, no acks received
May 21 12:16:18 ip-172-16-15-126 kernel: [335608.624799] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 845.
May 21 12:16:18 ip-172-16-15-126 kernel: [335608.629358] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 846.
May 21 12:16:18 ip-172-16-15-126 kernel: [335608.633809] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 847.
May 21 12:16:18 ip-172-16-15-126 kernel: [335608.638172] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 848.
May 21 12:16:18 ip-172-16-15-126 kernel: [335608.642539] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 849.
May 21 12:16:18 ip-172-16-15-126 kernel: [335608.646840] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 850.
May 21 12:16:18 ip-172-16-15-126 kernel: [335608.651040] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 851.
May 21 12:16:18 ip-172-16-15-126 kernel: [335608.655200] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 210.
May 21 12:16:18 ip-172-16-15-126 kernel: [335608.659342] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 211.
May 21 12:16:19 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:19 [INFO] memberlist: Marking i-09013c5adb8f3f726 as failed, suspect timeout reached
May 21 12:16:19 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:19 [INFO] serf: EventMemberFailed: i-09013c5adb8f3f726 172.16.16.225
May 21 12:16:19 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:19 [INFO] memberlist: Suspect i-0b10c1368ac9bd218 has failed, no acks received
May 21 12:16:19 ip-172-16-15-126 kernel: [335609.686391] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 937.
May 21 12:16:19 ip-172-16-15-126 kernel: [335609.691116] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 938.
May 21 12:16:19 ip-172-16-15-126 kernel: [335609.695751] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 939.
May 21 12:16:19 ip-172-16-15-126 kernel: [335609.700366] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 940.
May 21 12:16:19 ip-172-16-15-126 kernel: [335609.704947] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 451.
May 21 12:16:19 ip-172-16-15-126 kernel: [335609.709635] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 452.
May 21 12:16:19 ip-172-16-15-126 kernel: [335609.714111] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 184.
May 21 12:16:19 ip-172-16-15-126 kernel: [335609.718668] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 187.
May 21 12:16:19 ip-172-16-15-126 kernel: [335609.723125] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 189.
May 21 12:16:19 ip-172-16-15-126 kernel: [335609.727549] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 190.
May 21 12:16:19 ip-172-16-15-126 kernel: [335609.731936] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 193.
May 21 12:16:19 ip-172-16-15-126 kernel: [335609.736257] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 195.
May 21 12:16:19 ip-172-16-15-126 kernel: [335609.740642] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 197.
May 21 12:16:19 ip-172-16-15-126 kernel: [335609.745014] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 198.
May 21 12:16:19 ip-172-16-15-126 kernel: [335609.749349] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 199.
May 21 12:16:19 ip-172-16-15-126 kernel: [335609.753895] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 200.
May 21 12:16:19 ip-172-16-15-126 kernel: [335609.758412] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 202.
May 21 12:16:19 ip-172-16-15-126 kernel: [335609.762935] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 204.
May 21 12:16:19 ip-172-16-15-126 kernel: [335609.767428] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 211.
May 21 12:16:20 ip-172-16-15-126 kernel: [335609.771942] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 212.
May 21 12:16:20 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:20 [INFO] memberlist: Marking i-0552ac87d19ad1bad as failed, suspect timeout reached
May 21 12:16:20 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:20 [INFO] serf: EventMemberFailed: i-0552ac87d19ad1bad 172.16.16.153
May 21 12:16:20 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:20 [INFO] memberlist: Suspect i-06c86172cdacb3d1f has failed, no acks received
May 21 12:16:21 ip-172-16-15-126 kernel: [335610.806392] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 175.
May 21 12:16:21 ip-172-16-15-126 kernel: [335610.811407] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 180.
May 21 12:16:21 ip-172-16-15-126 kernel: [335610.816001] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 205.
May 21 12:16:21 ip-172-16-15-126 kernel: [335610.820605] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 206.
May 21 12:16:21 ip-172-16-15-126 kernel: [335610.825100] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 207.
May 21 12:16:21 ip-172-16-15-126 kernel: [335610.829034] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 208.
May 21 12:16:21 ip-172-16-15-126 kernel: [335610.832944] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 209.
May 21 12:16:21 ip-172-16-15-126 kernel: [335610.836835] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 210.
May 21 12:16:21 ip-172-16-15-126 kernel: [335610.840719] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 214.
May 21 12:16:21 ip-172-16-15-126 kernel: [335610.844627] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 215.
May 21 12:16:21 ip-172-16-15-126 kernel: [335610.848511] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 271.
May 21 12:16:21 ip-172-16-15-126 kernel: [335610.852405] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 272.
May 21 12:16:21 ip-172-16-15-126 kernel: [335610.856249] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 273.
May 21 12:16:21 ip-172-16-15-126 kernel: [335610.860091] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 274.
May 21 12:16:21 ip-172-16-15-126 kernel: [335610.863936] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 275.
May 21 12:16:21 ip-172-16-15-126 kernel: [335610.867754] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 276.
May 21 12:16:21 ip-172-16-15-126 kernel: [335610.871603] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 277.
May 21 12:16:21 ip-172-16-15-126 kernel: [335610.875487] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 278.
May 21 12:16:21 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:21 [INFO] memberlist: Marking i-06675422c58ce18ce as failed, suspect timeout reached
May 21 12:16:21 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:21 [INFO] serf: EventMemberFailed: i-06675422c58ce18ce 172.16.15.131
May 21 12:16:21 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:21 [INFO] memberlist: Suspect i-01b6b77ca63499415 has failed, no acks received
May 21 12:16:22 ip-172-16-15-126 kernel: [335611.894421] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 0, index 1016.
May 21 12:16:22 ip-172-16-15-126 kernel: [335611.898375] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 852.
May 21 12:16:22 ip-172-16-15-126 kernel: [335611.902204] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 853.
May 21 12:16:22 ip-172-16-15-126 kernel: [335611.906010] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 854.
May 21 12:16:22 ip-172-16-15-126 kernel: [335611.909805] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 856.
May 21 12:16:22 ip-172-16-15-126 kernel: [335611.913607] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 857.
May 21 12:16:22 ip-172-16-15-126 kernel: [335611.917583] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 858.
May 21 12:16:22 ip-172-16-15-126 kernel: [335611.921533] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 859.
May 21 12:16:22 ip-172-16-15-126 kernel: [335611.925438] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 860.
May 21 12:16:22 ip-172-16-15-126 kernel: [335611.929378] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 861.
May 21 12:16:22 ip-172-16-15-126 kernel: [335611.933288] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 212.
May 21 12:16:22 ip-172-16-15-126 kernel: [335611.937194] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 213.
May 21 12:16:22 ip-172-16-15-126 kernel: [335611.941101] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 214.
May 21 12:16:22 ip-172-16-15-126 kernel: [335611.944953] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 217.
May 21 12:16:22 ip-172-16-15-126 kernel: [335611.948899] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 218.
May 21 12:16:22 ip-172-16-15-126 kernel: [335611.952971] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 219.
May 21 12:16:22 ip-172-16-15-126 kernel: [335611.957173] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 551.
May 21 12:16:22 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:22 [INFO] memberlist: Marking i-0280f96a4564c8ccd as failed, suspect timeout reached
May 21 12:16:22 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:22 [INFO] serf: EventMemberFailed: i-0280f96a4564c8ccd 172.16.14.51
May 21 12:16:22 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:22 [INFO] memberlist: Suspect i-04cf3f2b71f631e5a has failed, no acks received
May 21 12:16:23 ip-172-16-15-126 kernel: [335612.982482] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 941.
May 21 12:16:23 ip-172-16-15-126 kernel: [335612.986922] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 453.
May 21 12:16:23 ip-172-16-15-126 kernel: [335612.991221] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 454.
May 21 12:16:23 ip-172-16-15-126 kernel: [335612.995516] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 186.
May 21 12:16:23 ip-172-16-15-126 kernel: [335612.999790] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 203.
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.004065] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 213.
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.008321] ena 0000:00:03.0 eth0: The number of lost tx completion is above the threshold (129 > 128). Reset the device
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.026469] ena 0000:00:03.0 eth0: Trigger reset is on
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.028757] ena 0000:00:03.0 eth0: tx_timeout: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.030839] ena 0000:00:03.0 eth0: io_suspend: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.032901] ena 0000:00:03.0 eth0: io_resume: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.034940] ena 0000:00:03.0 eth0: wd_expired: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.037010] ena 0000:00:03.0 eth0: interface_up: 1
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.039134] ena 0000:00:03.0 eth0: interface_down: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.041332] ena 0000:00:03.0 eth0: admin_q_pause: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.043485] ena 0000:00:03.0 eth0: queue_0_tx_cnt: 54560759
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.045893] ena 0000:00:03.0 eth0: queue_0_tx_bytes: 109427757914
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.048495] ena 0000:00:03.0 eth0: queue_0_tx_queue_stop: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.050982] ena 0000:00:03.0 eth0: queue_0_tx_queue_wakeup: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.053537] ena 0000:00:03.0 eth0: queue_0_tx_dma_mapping_err: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.056212] ena 0000:00:03.0 eth0: queue_0_tx_linearize: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.058706] ena 0000:00:03.0 eth0: queue_0_tx_linearize_failed: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.061394] ena 0000:00:03.0 eth0: queue_0_tx_napi_comp: 140035994
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.064115] ena 0000:00:03.0 eth0: queue_0_tx_tx_poll: 140035997
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.066821] ena 0000:00:03.0 eth0: queue_0_tx_doorbells: 46403547
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.069507] ena 0000:00:03.0 eth0: queue_0_tx_prepare_ctx_err: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.072137] ena 0000:00:03.0 eth0: queue_0_tx_bad_req_id: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.074634] ena 0000:00:03.0 eth0: queue_0_rx_cnt: 120515032
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.077147] ena 0000:00:03.0 eth0: queue_0_rx_bytes: 611095745645
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.079817] ena 0000:00:03.0 eth0: queue_0_rx_refil_partial: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.082434] ena 0000:00:03.0 eth0: queue_0_rx_bad_csum: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.084841] ena 0000:00:03.0 eth0: queue_0_rx_page_alloc_fail: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.087511] ena 0000:00:03.0 eth0: queue_0_rx_skb_alloc_fail: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.090134] ena 0000:00:03.0 eth0: queue_0_rx_dma_mapping_err: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.092775] ena 0000:00:03.0 eth0: queue_0_rx_bad_desc_num: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.095317] ena 0000:00:03.0 eth0: queue_0_rx_rx_copybreak_pkt: 18805624
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.098269] ena 0000:00:03.0 eth0: queue_1_tx_cnt: 34543466
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.100759] ena 0000:00:03.0 eth0: queue_1_tx_bytes: 90804732689
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.103396] ena 0000:00:03.0 eth0: queue_1_tx_queue_stop: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.105866] ena 0000:00:03.0 eth0: queue_1_tx_queue_wakeup: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.108407] ena 0000:00:03.0 eth0: queue_1_tx_dma_mapping_err: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.111057] ena 0000:00:03.0 eth0: queue_1_tx_linearize: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.113515] ena 0000:00:03.0 eth0: queue_1_tx_linearize_failed: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.116212] ena 0000:00:03.0 eth0: queue_1_tx_napi_comp: 79958517
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.118918] ena 0000:00:03.0 eth0: queue_1_tx_tx_poll: 79958518
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.121528] ena 0000:00:03.0 eth0: queue_1_tx_doorbells: 27611647
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.124197] ena 0000:00:03.0 eth0: queue_1_tx_prepare_ctx_err: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.126855] ena 0000:00:03.0 eth0: queue_1_tx_bad_req_id: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.129362] ena 0000:00:03.0 eth0: queue_1_rx_cnt: 62402699
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.131839] ena 0000:00:03.0 eth0: queue_1_rx_bytes: 288495267776
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.134539] ena 0000:00:03.0 eth0: queue_1_rx_refil_partial: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.137119] ena 0000:00:03.0 eth0: queue_1_rx_bad_csum: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.139532] ena 0000:00:03.0 eth0: queue_1_rx_page_alloc_fail: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.142167] ena 0000:00:03.0 eth0: queue_1_rx_skb_alloc_fail: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.144776] ena 0000:00:03.0 eth0: queue_1_rx_dma_mapping_err: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.147412] ena 0000:00:03.0 eth0: queue_1_rx_bad_desc_num: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.149994] ena 0000:00:03.0 eth0: queue_1_rx_rx_copybreak_pkt: 12404633
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.152903] ena 0000:00:03.0 eth0: queue_2_tx_cnt: 45547747
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.155387] ena 0000:00:03.0 eth0: queue_2_tx_bytes: 88046639394
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.158037] ena 0000:00:03.0 eth0: queue_2_tx_queue_stop: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.160529] ena 0000:00:03.0 eth0: queue_2_tx_queue_wakeup: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.163069] ena 0000:00:03.0 eth0: queue_2_tx_dma_mapping_err: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.165704] ena 0000:00:03.0 eth0: queue_2_tx_linearize: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.168150] ena 0000:00:03.0 eth0: queue_2_tx_linearize_failed: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.170836] ena 0000:00:03.0 eth0: queue_2_tx_napi_comp: 188870192
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.173538] ena 0000:00:03.0 eth0: queue_2_tx_tx_poll: 189138904
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.176199] ena 0000:00:03.0 eth0: queue_2_tx_doorbells: 38910085
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.178879] ena 0000:00:03.0 eth0: queue_2_tx_prepare_ctx_err: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.181550] ena 0000:00:03.0 eth0: queue_2_tx_bad_req_id: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.184019] ena 0000:00:03.0 eth0: queue_2_rx_cnt: 301701153
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.186534] ena 0000:00:03.0 eth0: queue_2_rx_bytes: 510479422410
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.189203] ena 0000:00:03.0 eth0: queue_2_rx_refil_partial: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.191789] ena 0000:00:03.0 eth0: queue_2_rx_bad_csum: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.194206] ena 0000:00:03.0 eth0: queue_2_rx_page_alloc_fail: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.196846] ena 0000:00:03.0 eth0: queue_2_rx_skb_alloc_fail: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.199453] ena 0000:00:03.0 eth0: queue_2_rx_dma_mapping_err: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.202085] ena 0000:00:03.0 eth0: queue_2_rx_bad_desc_num: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.204630] ena 0000:00:03.0 eth0: queue_2_rx_rx_copybreak_pkt: 10178455
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.207532] ena 0000:00:03.0 eth0: queue_3_tx_cnt: 33417773
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.210018] ena 0000:00:03.0 eth0: queue_3_tx_bytes: 91776922876
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.212668] ena 0000:00:03.0 eth0: queue_3_tx_queue_stop: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.215158] ena 0000:00:03.0 eth0: queue_3_tx_queue_wakeup: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.217710] ena 0000:00:03.0 eth0: queue_3_tx_dma_mapping_err: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.220355] ena 0000:00:03.0 eth0: queue_3_tx_linearize: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.222809] ena 0000:00:03.0 eth0: queue_3_tx_linearize_failed: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.225473] ena 0000:00:03.0 eth0: queue_3_tx_napi_comp: 72533647
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.228139] ena 0000:00:03.0 eth0: queue_3_tx_tx_poll: 72533647
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.230776] ena 0000:00:03.0 eth0: queue_3_tx_doorbells: 25996045
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.233450] ena 0000:00:03.0 eth0: queue_3_tx_prepare_ctx_err: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.236093] ena 0000:00:03.0 eth0: queue_3_tx_bad_req_id: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.238584] ena 0000:00:03.0 eth0: queue_3_rx_cnt: 58050897
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.241073] ena 0000:00:03.0 eth0: queue_3_rx_bytes: 291255301924
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.243745] ena 0000:00:03.0 eth0: queue_3_rx_refil_partial: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.246316] ena 0000:00:03.0 eth0: queue_3_rx_bad_csum: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.248725] ena 0000:00:03.0 eth0: queue_3_rx_page_alloc_fail: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.251406] ena 0000:00:03.0 eth0: queue_3_rx_skb_alloc_fail: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.254014] ena 0000:00:03.0 eth0: queue_3_rx_dma_mapping_err: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.256649] ena 0000:00:03.0 eth0: queue_3_rx_bad_desc_num: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.259243] ena 0000:00:03.0 eth0: queue_3_rx_rx_copybreak_pkt: 12242740
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.262194] ena 0000:00:03.0 eth0: queue_4_tx_cnt: 97417136
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.264660] ena 0000:00:03.0 eth0: queue_4_tx_bytes: 83594367057
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.267301] ena 0000:00:03.0 eth0: queue_4_tx_queue_stop: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.269779] ena 0000:00:03.0 eth0: queue_4_tx_queue_wakeup: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.272324] ena 0000:00:03.0 eth0: queue_4_tx_dma_mapping_err: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.274984] ena 0000:00:03.0 eth0: queue_4_tx_linearize: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.277440] ena 0000:00:03.0 eth0: queue_4_tx_linearize_failed: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.280142] ena 0000:00:03.0 eth0: queue_4_tx_napi_comp: 204885202
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.283215] ena 0000:00:03.0 eth0: queue_4_tx_tx_poll: 204885203
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.286199] ena 0000:00:03.0 eth0: queue_4_tx_doorbells: 91866448
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.289228] ena 0000:00:03.0 eth0: queue_4_tx_prepare_ctx_err: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.292209] ena 0000:00:03.0 eth0: queue_4_tx_bad_req_id: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.294725] ena 0000:00:03.0 eth0: queue_4_rx_cnt: 146586586
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.297239] ena 0000:00:03.0 eth0: queue_4_rx_bytes: 656436382527
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.299929] ena 0000:00:03.0 eth0: queue_4_rx_refil_partial: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.302513] ena 0000:00:03.0 eth0: queue_4_rx_bad_csum: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.304936] ena 0000:00:03.0 eth0: queue_4_rx_page_alloc_fail: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.307582] ena 0000:00:03.0 eth0: queue_4_rx_skb_alloc_fail: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.310194] ena 0000:00:03.0 eth0: queue_4_rx_dma_mapping_err: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.312846] ena 0000:00:03.0 eth0: queue_4_rx_bad_desc_num: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.315409] ena 0000:00:03.0 eth0: queue_4_rx_rx_copybreak_pkt: 27715663
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.318308] ena 0000:00:03.0 eth0: queue_5_tx_cnt: 54269388
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.320797] ena 0000:00:03.0 eth0: queue_5_tx_bytes: 95150507647
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.323441] ena 0000:00:03.0 eth0: queue_5_tx_queue_stop: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.325911] ena 0000:00:03.0 eth0: queue_5_tx_queue_wakeup: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.328456] ena 0000:00:03.0 eth0: queue_5_tx_dma_mapping_err: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.331109] ena 0000:00:03.0 eth0: queue_5_tx_linearize: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.333563] ena 0000:00:03.0 eth0: queue_5_tx_linearize_failed: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.336240] ena 0000:00:03.0 eth0: queue_5_tx_napi_comp: 111649875
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.338951] ena 0000:00:03.0 eth0: queue_5_tx_tx_poll: 111649875
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.341601] ena 0000:00:03.0 eth0: queue_5_tx_doorbells: 48317709
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.344272] ena 0000:00:03.0 eth0: queue_5_tx_prepare_ctx_err: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.346918] ena 0000:00:03.0 eth0: queue_5_tx_bad_req_id: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.349396] ena 0000:00:03.0 eth0: queue_5_rx_cnt: 77300362
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.351895] ena 0000:00:03.0 eth0: queue_5_rx_bytes: 326773596123
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.354578] ena 0000:00:03.0 eth0: queue_5_rx_refil_partial: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.357156] ena 0000:00:03.0 eth0: queue_5_rx_bad_csum: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.359595] ena 0000:00:03.0 eth0: queue_5_rx_page_alloc_fail: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.362279] ena 0000:00:03.0 eth0: queue_5_rx_skb_alloc_fail: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.364932] ena 0000:00:03.0 eth0: queue_5_rx_dma_mapping_err: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.367617] ena 0000:00:03.0 eth0: queue_5_rx_bad_desc_num: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.370200] ena 0000:00:03.0 eth0: queue_5_rx_rx_copybreak_pkt: 18377680
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.373122] ena 0000:00:03.0 eth0: queue_6_tx_cnt: 34298113
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.375628] ena 0000:00:03.0 eth0: queue_6_tx_bytes: 64209391587
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.378292] ena 0000:00:03.0 eth0: queue_6_tx_queue_stop: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.380798] ena 0000:00:03.0 eth0: queue_6_tx_queue_wakeup: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.383413] ena 0000:00:03.0 eth0: queue_6_tx_dma_mapping_err: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.386087] ena 0000:00:03.0 eth0: queue_6_tx_linearize: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.388576] ena 0000:00:03.0 eth0: queue_6_tx_linearize_failed: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.391285] ena 0000:00:03.0 eth0: queue_6_tx_napi_comp: 101257578
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.394031] ena 0000:00:03.0 eth0: queue_6_tx_tx_poll: 101257578
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.396711] ena 0000:00:03.0 eth0: queue_6_tx_doorbells: 30137999
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.399417] ena 0000:00:03.0 eth0: queue_6_tx_prepare_ctx_err: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.402083] ena 0000:00:03.0 eth0: queue_6_tx_bad_req_id: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.404597] ena 0000:00:03.0 eth0: queue_6_rx_cnt: 87915741
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.407142] ena 0000:00:03.0 eth0: queue_6_rx_bytes: 352158714788
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.409846] ena 0000:00:03.0 eth0: queue_6_rx_refil_partial: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.412446] ena 0000:00:03.0 eth0: queue_6_rx_bad_csum: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.414918] ena 0000:00:03.0 eth0: queue_6_rx_page_alloc_fail: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.417613] ena 0000:00:03.0 eth0: queue_6_rx_skb_alloc_fail: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.420262] ena 0000:00:03.0 eth0: queue_6_rx_dma_mapping_err: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.422927] ena 0000:00:03.0 eth0: queue_6_rx_bad_desc_num: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.425486] ena 0000:00:03.0 eth0: queue_6_rx_rx_copybreak_pkt: 20796029
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.428442] ena 0000:00:03.0 eth0: queue_7_tx_cnt: 52485410
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.430976] ena 0000:00:03.0 eth0: queue_7_tx_bytes: 67669867898
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.433672] ena 0000:00:03.0 eth0: queue_7_tx_queue_stop: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.436175] ena 0000:00:03.0 eth0: queue_7_tx_queue_wakeup: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.438750] ena 0000:00:03.0 eth0: queue_7_tx_dma_mapping_err: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.441428] ena 0000:00:03.0 eth0: queue_7_tx_linearize: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.443895] ena 0000:00:03.0 eth0: queue_7_tx_linearize_failed: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.446612] ena 0000:00:03.0 eth0: queue_7_tx_napi_comp: 153127460
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.449352] ena 0000:00:03.0 eth0: queue_7_tx_tx_poll: 153127463
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.452053] ena 0000:00:03.0 eth0: queue_7_tx_doorbells: 48526263
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.454758] ena 0000:00:03.0 eth0: queue_7_tx_prepare_ctx_err: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.457433] ena 0000:00:03.0 eth0: queue_7_tx_bad_req_id: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.459963] ena 0000:00:03.0 eth0: queue_7_rx_cnt: 140231578
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.462513] ena 0000:00:03.0 eth0: queue_7_rx_bytes: 674949155576
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.465205] ena 0000:00:03.0 eth0: queue_7_rx_refil_partial: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.467806] ena 0000:00:03.0 eth0: queue_7_rx_bad_csum: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.470275] ena 0000:00:03.0 eth0: queue_7_rx_page_alloc_fail: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.472931] ena 0000:00:03.0 eth0: queue_7_rx_skb_alloc_fail: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.475581] ena 0000:00:03.0 eth0: queue_7_rx_dma_mapping_err: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.478254] ena 0000:00:03.0 eth0: queue_7_rx_bad_desc_num: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.480832] ena 0000:00:03.0 eth0: queue_7_rx_rx_copybreak_pkt: 24570422
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.483797] ena 0000:00:03.0 eth0: ena_admin_q_aborted_cmd: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.486375] ena 0000:00:03.0 eth0: ena_admin_q_submitted_cmd: 44
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.489046] ena 0000:00:03.0 eth0: ena_admin_q_completed_cmd: 44
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.505136] ena 0000:00:03.0 eth0: ena_admin_q_out_of_space: 0
May 21 12:16:23 ip-172-16-15-126 kernel: [335613.507742] ena 0000:00:03.0 eth0: ena_admin_q_no_completion: 0
May 21 12:16:23 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:23 [INFO] memberlist: Marking i-03c4d88e2854d047b as failed, suspect timeout reached
May 21 12:16:23 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:23 [INFO] serf: EventMemberFailed: i-03c4d88e2854d047b 172.16.16.159
May 21 12:16:23 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:23 [INFO] memberlist: Suspect i-0caf05d83d7af272a has failed, no acks received
May 21 12:16:24 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:24 [INFO] memberlist: Marking i-0af21f21158ce7c92 as failed, suspect timeout reached
May 21 12:16:24 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:24 [INFO] serf: EventMemberFailed: i-0af21f21158ce7c92 172.16.14.227
May 21 12:16:24 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:24 [INFO] memberlist: Suspect i-0d2a89cac90d1b339 has failed, no acks received
May 21 12:16:25 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:25 [INFO] memberlist: Marking i-0f14d98ae903e7de2 as failed, suspect timeout reached
May 21 12:16:25 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:25 [INFO] serf: EventMemberFailed: i-0f14d98ae903e7de2 172.16.14.67
May 21 12:16:25 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:25 [INFO] consul: removing server i-0f14d98ae903e7de2 (Addr: 172.16.14.67:8300) (DC: dc1)
May 21 12:16:25 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:25 [INFO] memberlist: Suspect i-07d37224cc2e12f36 has failed, no acks received
May 21 12:16:25 ip-172-16-15-126 kernel: [335615.670560] ena: Reset indication didn't turn off
May 21 12:16:25 ip-172-16-15-126 kernel: [335615.673353] ena 0000:00:03.0: Device reset failed
May 21 12:16:25 ip-172-16-15-126 kernel: [335615.676424] ena 0000:00:03.0 eth0: free uncompleted tx skb qid 0 idx 0x3e1
May 21 12:16:25 ip-172-16-15-126 kernel: [335615.682126] ena 0000:00:03.0 eth0: free uncompleted tx skb qid 1 idx 0x329
May 21 12:16:25 ip-172-16-15-126 kernel: [335615.687797] ena 0000:00:03.0 eth0: free uncompleted tx skb qid 2 idx 0xab
May 21 12:16:25 ip-172-16-15-126 kernel: [335615.693431] ena 0000:00:03.0 eth0: free uncompleted tx skb qid 3 idx 0x20d
May 21 12:16:25 ip-172-16-15-126 kernel: [335615.699073] ena 0000:00:03.0 eth0: free uncompleted tx skb qid 4 idx 0x389
May 21 12:16:25 ip-172-16-15-126 kernel: [335615.704681] ena 0000:00:03.0 eth0: free uncompleted tx skb qid 5 idx 0x1a0
May 21 12:16:25 ip-172-16-15-126 kernel: [335615.710284] ena 0000:00:03.0 eth0: free uncompleted tx skb qid 6 idx 0x15
May 21 12:16:25 ip-172-16-15-126 kernel: [335615.715969] ena 0000:00:03.0 eth0: free uncompleted tx skb qid 7 idx 0xed
May 21 12:16:26 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:26 [ERR] agent: coordinate update error: rpc error: failed to get conn: dial tcp 172.16.14.66:8300: getsockopt: no route to host
May 21 12:16:26 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:26 [INFO] memberlist: Marking i-02b2968ab933cf778 as failed, suspect timeout reached
May 21 12:16:26 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:26 [INFO] serf: EventMemberFailed: i-02b2968ab933cf778 172.16.15.32
May 21 12:16:26 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:26 [INFO] memberlist: Suspect i-0edd0cf9d6f372a8c has failed, no acks received
May 21 12:16:27 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:27 [INFO] memberlist: Marking i-055abe2ce793ae2c8 as failed, suspect timeout reached
May 21 12:16:27 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:27 [INFO] serf: EventMemberFailed: i-055abe2ce793ae2c8 172.16.14.149
May 21 12:16:27 ip-172-16-15-126 consul[3614]: 2017/05/21 12:16:27 [INFO] memberlist: Suspect i-001ba44e32334f963 has failed, no acks received
-- snip -- lost logs, weren't stored in /var/log/messages, but recovered from console: --
_bm_flush+0x1c/0x20 [dm_persistent_data]
[365557.729461] [<ffffffffa08bffc3>] dm_tm_commit+0x33/0x40 [dm_persistent_data]
[365557.735149] [<ffffffffa08d8d39>] __commit_transaction+0x359/0x360 [dm_thin_pool]
[365557.740735] [<ffffffffa08d9f47>] dm_pool_commit_metadata+0x37/0x60 [dm_thin_pool]
[365557.746227] [<ffffffffa08d3967>] commit+0x27/0x90 [dm_thin_pool]
[365557.750505] [<ffffffffa08d3cf2>] pool_status+0x322/0x6b0 [dm_thin_pool]
[365557.755227] [<ffffffff811cd5c6>] ? __memcg_kmem_charge+0x46/0x90
[365557.759749] [<ffffffffa0889c96>] retrieve_status+0xa6/0x1b0 [dm_mod]
[365557.764474] [<ffffffffa088ae14>] table_status+0x54/0x80 [dm_mod]
[365557.768939] [<ffffffffa088adc0>] ? dm_get_live_or_inactive_table.isra.3+0x30/0x30 [dm_mod]
[365557.775238] [<ffffffffa088aae6>] ctl_ioctl+0x246/0x4b0 [dm_mod]
[365557.778569] [<ffffffff8107932b>] ? recalc_sigpending+0x1b/0x50
[365557.781825] [<ffffffffa088ad63>] dm_ctl_ioctl+0x13/0x20 [dm_mod]
[365557.785157] [<ffffffff811e772d>] do_vfs_ioctl+0x2cd/0x4b0
[365557.788226] [<ffffffff8107c27b>] ? __set_current_blocked+0x4b/0x60
[365557.791642] [<ffffffff811e7989>] SyS_ioctl+0x79/0x90
[365557.794518] [<ffffffff8107c4a7>] ? SyS_rt_sigprocmask+0x67/0xb0
[365557.797936] [<ffffffff814eff6e>] entry_SYSCALL_64_fastpath+0x12/0x71
[365557.801516] INFO: task rs:main Q:Reg:3401 blocked for more than 120 seconds.
[365557.805308] Tainted: G E 4.4.51-40.58.amzn1.x86_64 #1
[365557.808942] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[365557.813403] rs:main Q:Reg D ffff881e6a3bf968 0 3401 1 0x00000000
[365557.817403] ffff881e6a3bf968 ffff881e77ab0000 ffff881e71ddba80 ffff881e6a3c0000
[365557.822043] 0000000000000000 7fffffffffffffff ffff881efff92a08 ffffffff814ecd30
[365557.840036] ffff881e6a3bf980 ffffffff814ec5f5 ffff881e84695340 ffff881e6a3bfa20
[365557.844684] Call Trace:
[365557.846326] [<ffffffff814ecd30>] ? bit_wait+0x50/0x50
[365557.850013] [<ffffffff814ec5f5>] schedule+0x35/0x80
[365557.853482] [<ffffffff814eef01>] schedule_timeout+0x231/0x2b0
[365557.857646] [<ffffffff8142c315>] ? netif_rx_internal+0x35/0xf0
[365557.861887] [<ffffffff8100c365>] ? xen_clocksource_get_cycles+0x15/0x20
[365557.866587] [<ffffffff810d2f7a>] ? ktime_get+0x3a/0x90
[365557.870317] [<ffffffff814ecd30>] ? bit_wait+0x50/0x50
[365557.874015] [<ffffffff814ebcf6>] io_schedule_timeout+0xa6/0x110
[365557.878223] [<ffffffff814ecd4b>] bit_wait_io+0x1b/0x60
[365557.881947] [<ffffffff814ec990>] __wait_on_bit+0x60/0x90
[365557.885801] [<ffffffff814ecd30>] ? bit_wait+0x50/0x50
[365557.889444] [<ffffffff814eca32>] out_of_line_wait_on_bit+0x72/0x80
[365557.893829] [<ffffffff810acf20>] ? autoremove_wake_function+0x40/0x40
[365557.898380] [<ffffffffa00ccdb9>] do_get_write_access+0x279/0x4c0 [jbd2]
[365557.903081] [<ffffffff8120918e>] ? __find_get_block+0xae/0x120
[365557.907299] [<ffffffffa00cd031>] jbd2_journal_get_write_access+0x31/0x60 [jbd2]
[365557.912781] [<ffffffffa0136511>] __ext4_journal_get_write_access+0x31/0x70 [ext4]
[365557.918323] [<ffffffffa01080da>] ext4_reserve_inode_write+0x6a/0x90 [ext4]
[365557.923190] [<ffffffffa010b888>] ? ext4_dirty_inode+0x48/0x70 [ext4]
[365557.927728] [<ffffffffa010813f>] ext4_mark_inode_dirty+0x3f/0x1f0 [ext4]
[365557.932491] [<ffffffffa010b888>] ext4_dirty_inode+0x48/0x70 [ext4]
[365557.936911] [<ffffffff812009e1>] __mark_inode_dirty+0x181/0x370
[365557.941163] [<ffffffff811ee887>] generic_update_time+0x87/0xd0
[365557.945363] [<ffffffff811edfd9>] file_update_time+0x99/0xf0
[365557.949398] [<ffffffff810dc784>] ? get_futex_key+0x194/0x270
[365557.953487] [<ffffffff81163f1c>] __generic_file_write_iter+0x9c/0x1e0
[365557.958070] [<ffffffff810dcd41>] ? futex_wake+0x81/0x150
[365557.961918] [<ffffffffa00feb76>] ext4_file_write_iter+0xf6/0x360 [ext4]
[365557.966561] [<ffffffff810dec50>] ? do_futex+0x100/0xad0
[365557.970384] [<ffffffff81411c68>] ? sock_sendmsg+0x38/0x50
[365557.974311] [<ffffffff814120e9>] ? SYSC_sendto+0x139/0x170
[365557.978286] [<ffffffff811d4dba>] __vfs_write+0xaa/0xe0
[365557.982034] [<ffffffff811d53e2>] vfs_write+0xa2/0x1a0
[365557.985716] [<ffffffff811d60e6>] SyS_write+0x46/0xa0
[365557.989368] [<ffffffff814eff6e>] entry_SYSCALL_64_fastpath+0x12/0x71
[365557.993848] INFO: task agent:43861 blocked for more than 120 seconds.
[365557.998607] Tainted: G E 4.4.51-40.58.amzn1.x86_64 #1
[365558.003736] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[365558.009987] agent D ffff880c4fdf7860 0 43861 13580 0x00000000
[365558.015615] ffff880c4fdf7860 ffff881e77ab0000 ffff881d07de57c0 ffff880c4fdf8000
[365558.022129] ffff881e70ff6558 ffff881e70ff6570 ffff880c4fdf7990 0000000000000000
[365558.028637] ffff880c4fdf7878 ffffffff814ec5f5 ffff881d07de57c0 ffff880c4fdf78c8
[365558.035142] Call Trace:
[365558.037302] [<ffffffff814ec5f5>] schedule+0x35/0x80
[365558.041067] [<ffffffff814eec2a>] rwsem_down_read_failed+0xea/0x120
[365558.045637] [<ffffffff812dabd4>] call_rwsem_down_read_failed+0x14/0x30
[365558.050637] [<ffffffff814ee2d7>] ? down_read+0x17/0x20
[365558.054694] [<ffffffffa08d98ae>] dm_thin_find_block+0x3e/0xd0 [dm_thin_pool]
[365558.060370] [<ffffffffa08d55cc>] thin_map+0x13c/0x280 [dm_thin_pool]
[365558.065245] [<ffffffffa0881eca>] __map_bio+0x3a/0x110 [dm_mod]
[365558.069759] [<ffffffffa0883630>] __split_and_process_bio+0x240/0x3c0 [dm_mod]
[365558.075500] [<ffffffffa088381a>] dm_make_request+0x6a/0xd0 [dm_mod]
[365558.080323] [<ffffffff812aa0c4>] generic_make_request+0xe4/0x1a0
[365558.084977] [<ffffffff812aa1e8>] submit_bio+0x68/0x130
[365558.089021] [<ffffffffa014d6e5>] ext4_mpage_readpages+0x2c5/0x7c0 [ext4]
[365558.094118] [<ffffffff811af068>] ? alloc_pages_current+0x88/0x120
[365558.098800] [<ffffffffa0103307>] ext4_readpages+0x37/0x40 [ext4]
[365558.103485] [<ffffffff8116e484>] __do_page_cache_readahead+0x174/0x200
[365558.108597] [<ffffffff81164714>] filemap_fault+0x334/0x3e0
[365558.113007] [<ffffffffa010bed6>] ext4_filemap_fault+0x36/0x50 [ext4]
[365558.118027] [<ffffffff8118bfbd>] __do_fault+0x3d/0x70
[365558.122134] [<ffffffff811903f7>] handle_mm_fault+0xf27/0x1870
[365558.126725] [<ffffffff810b3501>] ? __raw_callee_save___pv_queued_spin_unlock+0x11/0x20
[365558.133188] [<ffffffff8105ed33>] __do_page_fault+0x183/0x3f0
[365558.137660] [<ffffffff8100c365>] ? xen_clocksource_get_cycles+0x15/0x20
[365558.142909] [<ffffffff8105efc2>] do_page_fault+0x22/0x30
[365558.147324] [<ffffffff814f2058>] page_fault+0x28/0x30
[365558.151830] INFO: task consul:15336 blocked for more than 120 seconds.
[365558.157299] Tainted: G E 4.4.51-40.58.amzn1.x86_64 #1
[365558.162777] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[365558.169393] consul D ffff881e4f6a3860 0 15336 15293 0x00000000
[365558.175451] ffff881e4f6a3860 ffffffff81a11500 ffff881e49d2ba80 ffff881e4f6a4000
[365558.182413] ffff881e70ff6558 ffff881e70ff6570 ffff881e4f6a3990 0000000000000000
[365558.189346] ffff881e4f6a3878 ffffffff814ec5f5 ffff881e49d2ba80 ffff881e4f6a38c8
[365558.196075] Call Trace:
[365558.198259] [<ffffffff814ec5f5>] schedule+0x35/0x80
[365558.202177] [<ffffffff814eec2a>] rwsem_down_read_failed+0xea/0x120
[365558.206997] [<ffffffff812dabd4>] call_rwsem_down_read_failed+0x14/0x30
[365558.212061] [<ffffffff814ee2d7>] ? down_read+0x17/0x20
[365558.216155] [<ffffffffa08d98ae>] dm_thin_find_block+0x3e/0xd0 [dm_thin_pool]
[365558.221855] [<ffffffffa08d55cc>] thin_map+0x13c/0x280 [dm_thin_pool]
[365558.226781] [<ffffffffa0881eca>] __map_bio+0x3a/0x110 [dm_mod]
[365558.231363] [<ffffffffa0883630>] __split_and_process_bio+0x240/0x3c0 [dm_mod]
[365558.237145] [<ffffffffa088381a>] dm_make_request+0x6a/0xd0 [dm_mod]
[365558.241819] [<ffffffff812aa0c4>] generic_make_request+0xe4/0x1a0
[365558.246294] [<ffffffff812aa1e8>] submit_bio+0x68/0x130
[365558.250354] [<ffffffff8117028e>] ? lru_cache_add+0xe/0x10
[365558.254594] [<ffffffff81162c1d>] ? add_to_page_cache_lru+0x4d/0x80
[365558.259375] [<ffffffffa014d6e5>] ext4_mpage_readpages+0x2c5/0x7c0 [ext4]
[365558.264526] [<ffffffff811af068>] ? alloc_pages_current+0x88/0x120
[365558.269243] [<ffffffffa0103307>] ext4_readpages+0x37/0x40 [ext4]
[365558.273910] [<ffffffff8116e484>] __do_page_cache_readahead+0x174/0x200
[365558.278920] [<ffffffff81164714>] filemap_fault+0x334/0x3e0
[365558.283247] [<ffffffffa010bed6>] ext4_filemap_fault+0x36/0x50 [ext4]
[365558.288152] [<ffffffff8118bfbd>] __do_fault+0x3d/0x70
[365558.292114] [<ffffffff811903f7>] handle_mm_fault+0xf27/0x1870
[365558.296529] [<ffffffff8100c365>] ? xen_clocksource_get_cycles+0x15/0x20
[365558.301583] [<ffffffff8105ed33>] __do_page_fault+0x183/0x3f0
[365558.306036] [<ffffffff8105efc2>] do_page_fault+0x22/0x30
[365558.310156] [<ffffffff814f2058>] page_fault+0x28/0x30
[365558.314101] INFO: task consul:15339 blocked for more than 120 seconds.
[365558.318987] Tainted: G E 4.4.51-40.58.amzn1.x86_64 #1
[365558.324015] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[365558.330096] consul D ffff881e4f69bd98 0 15339 15293 0x00000000
[365558.335620] ffff881e4f69bd98 ffff881e77a9ba80 ffff881e51918000 ffff881e4f69c000
[365558.341683] ffff881e749dbc80 ffff881e749dbc68 ffffffff00000000 ffffffff00000002
[365558.347735] ffff881e4f69bdb0 ffffffff814ec5f5 ffff881e51918000 ffff881e4f69be30
[365558.354237] Call Trace:
[365558.356452] [<ffffffff814ec5f5>] schedule+0x35/0x80
[365558.360397] [<ffffffff814ee9fa>] rwsem_down_write_failed+0x1da/0x320
[365558.365314] [<ffffffff810a03b5>] ? pick_next_entity+0xa5/0x160
[365558.369767] [<ffffffff812dac03>] call_rwsem_down_write_failed+0x13/0x20
[365558.374620] [<ffffffff814ee304>] ? down_write+0x24/0x40
[365558.378673] [<ffffffff8117d53b>] vm_mmap_pgoff+0x6b/0xc0
[365558.382693] [<ffffffff811955b3>] SyS_mmap_pgoff+0x103/0x260
[365558.386914] [<ffffffff810d240a>] ? ktime_get_ts64+0x4a/0xe0
[365558.391291] [<ffffffff8101b292>] SyS_mmap+0x22/0x30
[365558.395186] [<ffffffff814eff6e>] entry_SYSCALL_64_fastpath+0x12/0x71
[365558.400120] INFO: task consul:15340 blocked for more than 120 seconds.
[365558.405105] Tainted: G E 4.4.51-40.58.amzn1.x86_64 #1
[365558.410223] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[365558.416474] consul D ffff881e4f657860 0 15340 15293 0x00000000
[365558.422156] ffff881e4f657860 ffff881e77aeba80 ffff881e69c9ba80 ffff881e4f658000
[365558.428666] ffff881e70ff6558 ffff881e70ff6570 ffff881e4f657990 0000000000000000
[365558.435155] ffff881e4f657878 ffffffff814ec5f5 ffff881e69c9ba80 ffff881e4f6578c8
[365558.441440] Call Trace:
[365558.443518] [<ffffffff814ec5f5>] schedule+0x35/0x80
[365558.447315] [<ffffffff814eec2a>] rwsem_down_read_failed+0xea/0x120
[365558.452132] [<ffffffff812dabd4>] call_rwsem_down_read_failed+0x14/0x30
[365558.457138] [<ffffffff814ee2d7>] ? down_read+0x17/0x20
[365558.461221] [<ffffffffa08d98ae>] dm_thin_find_block+0x3e/0xd0 [dm_thin_pool]
[365558.466932] [<ffffffffa08d55cc>] thin_map+0x13c/0x280 [dm_thin_pool]
[365558.471825] [<ffffffffa0881eca>] __map_bio+0x3a/0x110 [dm_mod]
[365558.476386] [<ffffffffa0883630>] __split_and_process_bio+0x240/0x3c0 [dm_mod]
[365558.482138] [<ffffffffa088381a>] dm_make_request+0x6a/0xd0 [dm_mod]
[365558.486966] [<ffffffff812aa0c4>] generic_make_request+0xe4/0x1a0
[365558.491617] [<ffffffff812aa1e8>] submit_bio+0x68/0x130
[365558.495686] [<ffffffff8117028e>] ? lru_cache_add+0xe/0x10
[365558.499933] [<ffffffff81162c1d>] ? add_to_page_cache_lru+0x4d/0x80
[365558.504756] [<ffffffffa014d6e5>] ext4_mpage_readpages+0x2c5/0x7c0 [ext4]
[365558.509903] [<ffffffff811af068>] ? alloc_pages_current+0x88/0x120
[365558.514668] [<ffffffffa0103307>] ext4_readpages+0x37/0x40 [ext4]
[365558.519387] [<ffffffff8116e484>] __do_page_cache_readahead+0x174/0x200
[365558.524455] [<ffffffff81164714>] filemap_fault+0x334/0x3e0
[365558.528782] [<ffffffffa010bed6>] ext4_filemap_fault+0x36/0x50 [ext4]
[365558.533708] [<ffffffff8118bfbd>] __do_fault+0x3d/0x70
[365558.537682] [<ffffffff811903f7>] handle_mm_fault+0xf27/0x1870
[365558.541997] [<ffffffff8147670a>] ? tcp_poll+0x6a/0x240
[365558.545909] [<ffffffff810b3501>] ? __raw_callee_save___pv_queued_spin_unlock+0x11/0x20
[365558.552179] [<ffffffff8121a631>] ? SyS_epoll_ctl+0xd1/0xca0
[365558.556523] [<ffffffff8105ed33>] __do_page_fault+0x183/0x3f0
[365558.574325] [<ffffffff8100c365>] ? xen_clocksource_get_cycles+0x15/0x20
[365558.579398] [<ffffffff8105efc2>] do_page_fault+0x22/0x30
[365558.583584] [<ffffffff814f2058>] page_fault+0x28/0x30
[365558.587630] INFO: task jbd2/dm-10-8:16273 blocked for more than 120 seconds.
[365558.593052] Tainted: G E 4.4.51-40.58.amzn1.x86_64 #1
[365558.598254] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[365558.604597] jbd2/dm-10-8 D ffff881df5e17b08 0 16273 2 0x00000000
[365558.610381] ffff881df5e17b08 ffffffff81a11500 ffff881e51bcba80 ffff881df5e18000
[365558.617026] 0000000000000000 7fffffffffffffff ffff881efff94478 ffffffff814ecd30
[365558.623543] ffff881df5e17b20 ffffffff814ec5f5 ffff881e84615340 ffff881df5e17bb8
[365558.630097] Call Trace:
[365558.632304] [<ffffffff814ecd30>] ? bit_wait+0x50/0x50
[365558.636368] [<ffffffff814ec5f5>] schedule+0x35/0x80
[365558.640199] [<ffffffff814eef01>] schedule_timeout+0x231/0x2b0
[365558.644499] [<ffffffff8100c365>] ? xen_clocksource_get_cycles+0x15/0x20
[365558.649548] [<ffffffff810d2f7a>] ? ktime_get+0x3a/0x90
[365558.653643] [<ffffffff814ecd30>] ? bit_wait+0x50/0x50
[365558.657688] [<ffffffff814ebcf6>] io_schedule_timeout+0xa6/0x110
[365558.662327] [<ffffffff814ecd4b>] bit_wait_io+0x1b/0x60
[365558.666432] [<ffffffff814ec990>] __wait_on_bit+0x60/0x90
[365558.670655] [<ffffffff814ecd30>] ? bit_wait+0x50/0x50
[365558.674676] [<ffffffff814eca32>] out_of_line_wait_on_bit+0x72/0x80
[365558.679505] [<ffffffff810acf20>] ? autoremove_wake_function+0x40/0x40
[365558.684492] [<ffffffff812088c7>] __wait_on_buffer+0x27/0x30
[365558.688881] [<ffffffffa00cf2f6>] jbd2_journal_commit_transaction+0xf66/0x1bb0 [jbd2]
[365558.695040] [<ffffffffa00d3c90>] kjournald2+0x120/0x340 [jbd2]
[365558.699622] [<ffffffff810acee0>] ? prepare_to_wait_event+0xf0/0xf0
[365558.704476] [<ffffffffa00d3b70>] ? commit_timeout+0x10/0x10 [jbd2]
[365558.709312] [<ffffffff8108a7f9>] kthread+0xc9/0xe0
[365558.713179] [<ffffffff8108a730>] ? kthread_park+0x60/0x60
[365558.717631] [<ffffffff814f02cf>] ret_from_fork+0x3f/0x70
[365558.721865] [<ffffffff8108a730>] ? kthread_park+0x60/0x60
[365558.726156] INFO: task java:16402 blocked for more than 120 seconds.
[365558.731055] Tainted: G E 4.4.51-40.58.amzn1.x86_64 #1
[365558.736209] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[365558.742194] java D ffff881e6bc37e08 0 16402 16351 0x00000000
[365558.747635] ffff881e6bc37e08 ffff881e77a99d40 ffff881e61288000 ffff881e6bc38000
[365558.754131] ffff881e75cd9340 ffff881e75cd9328 ffffffff00000000 ffffffff00000001
[365558.760631] ffff881e6bc37e20 ffffffff814ec5f5 ffff881e61288000 ffff881e6bc37ea0
[365558.767086] Call Trace:
[365558.769269] [<ffffffff814ec5f5>] schedule+0x35/0x80
[365558.773139] [<ffffffff814ee9fa>] rwsem_down_write_failed+0x1da/0x320
[365558.778030] [<ffffffff812dac03>] call_rwsem_down_write_failed+0x13/0x20
[365558.783083] [<ffffffff814ee304>] ? down_write+0x24/0x40
[365558.787210] [<ffffffff811987d1>] SyS_mprotect+0xc1/0x240
[365558.791414] [<ffffffff810cefd1>] ? posix_ktime_get_ts+0x11/0x20
[365558.796010] [<ffffffff814eff6e>] entry_SYSCALL_64_fastpath+0x12/0x71
[365558.800940] INFO: task java:16445 blocked for more than 120 seconds.
[365558.805799] Tainted: G E 4.4.51-40.58.amzn1.x86_64 #1
[365558.810897] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[365558.817115] java D ffff881debbd3e10 0 16445 16351 0x00000000
[365558.822756] ffff881debbd3e10 ffffffff81a11500 ffff881e6bd2ba80 ffff881debbd4000
[365558.829278] ffff881e75cd9328 ffff881e75cd9340 ffff881debbd3f58 ffff881e6bd2ba80
[365558.835788] ffff881debbd3e28 ffffffff814ec5f5 ffff881e6bd2ba80 ffff881debbd3e78
[365558.842069] Call Trace:
[365558.844150] [<ffffffff814ec5f5>] schedule+0x35/0x80
[365558.847945] [<ffffffff814eec2a>] rwsem_down_read_failed+0xea/0x120
[365558.852755] [<ffffffff812dabd4>] call_rwsem_down_read_failed+0x14/0x30
[365558.857772] [<ffffffff814ee2d7>] ? down_read+0x17/0x20
[365558.861861] [<ffffffff8105ef39>] __do_page_fault+0x389/0x3f0
[365558.866301] [<ffffffff8100c365>] ? xen_clocksource_get_cycles+0x15/0x20
[365558.871390] [<ffffffff8105efc2>] do_page_fault+0x22/0x30
[365558.875588] [<ffffffff814f2058>] page_fault+0x28/0x30
[365665.548153] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 858.
[365703.557562] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 229.
[365704.561666] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 567.
[365704.566122] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 1021.
[365705.569628] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 647.
[365705.573997] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 648.
[365706.577713] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 230.
[365707.581763] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 876.
[365708.585837] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 649.
[365709.593807] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 809.
[365709.598182] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 231.
[365711.601901] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 650.
[365712.605951] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 810.
[365714.610007] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 651.
[365724.614383] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 232.
[365730.618610] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 0, index 87.
[365737.622898] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 872.
[365739.626961] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 839.
[365742.631063] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 0, index 88.
[365742.635070] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 0, index 89.
[365742.639040] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 840.
[365742.643015] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 841.
[365742.647046] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 811.
[365742.651030] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 812.
[365742.655302] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 813.
[365742.659545] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 814.
[365742.663826] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 815.
[365742.668055] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 233.
[365743.671120] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 1022.
[365743.675192] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 1023.
[365744.683098] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 652.
[365744.687117] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 653.
[365746.691242] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 0.
[365747.695279] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 654.
[365748.699313] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 842.
[365749.703345] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 568.
[365749.707929] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 569.
[365749.712383] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 1.
[365749.716774] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 2.
[365763.723946] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 816.
[365894.752829] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 899.
[366004.772985] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 14.
[366006.777073] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 849.
[366007.781113] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 15.
[366007.785109] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 911.
[366012.789315] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 850.
[366013.793349] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 16.
[366025.797802] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 919.
[366027.801877] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 248.
[366030.805996] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 1, index 851.
[366031.810023] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 586.
[366031.814046] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 587.
[366044.830516] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 669.
[366060.835104] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 0, index 102.
[366060.839915] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 822.
[366061.843146] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 588.
[366063.847294] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 3, index 249.
[366072.863541] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 823.
[366074.863683] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 924.
[366100.868637] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 589.
[366103.872750] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 17.
[366103.876745] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 18.
[366105.880828] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 824.
[366105.884859] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 825.
[366109.888993] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 5, index 23.
[366117.893301] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 2, index 827.
[366283.903580] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 6, index 962.
[366305.908408] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 682.
[366313.912713] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 596.
[366313.917037] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 4, index 597.
[366314.920749] ena 0000:00:03.0 eth0: Found a Tx that wasn't completed on time, qid 7, index 683.
[366377.372692] EXT4-fs (dm-8): mounted filesystem with ordered data mode. Opts: (null)
[366377.602016] EXT4-fs (dm-8): mounted filesystem with ordered data mode. Opts: (null)
[366377.782775] EXT4-fs (dm-8): mounted filesystem with ordered data mode. Opts: (null)
[391563.883504] EXT4-fs (dm-6): mounted filesystem with ordered data mode. Opts: (null)
[391564.067606] EXT4-fs (dm-6): mounted filesystem with ordered data mode. Opts: (null)
[391564.222983] EXT4-fs (dm-6): mounted filesystem with ordered data mode. Opts: (null)
[395856.136657] audit: type=1305 audit(1495033308.746:1058): audit_pid=0 old=3379 auid=4294967295 ses=4294967295 res=1
[395856.229645] audit: type=1305 audit(1495033308.838:1059): audit_enabled=0 old=1 auid=500 ses=91 res=1
[395856.251799] audit: type=1305 audit(1495033308.862:1060): audit_pid=109278 old=0 auid=500 ses=91 res=1
/dev/fd/10: line 1: /sbin/plymouthd: No such file or directory
initctl: Event failed
Stopping docker: [395889.810592] docker0: port 1(vethd31d3a8) entered disabled state
[395889.813485] veth6d92c3f: renamed from eth0
[395889.874456] docker0: port 1(vethd31d3a8) entered disabled state
[395889.880996] device vethd31d3a8 left promiscuous mode
[395889.883901] docker0: port 1(vethd31d3a8) entered disabled state
[ OK ]
Stopping sshd: [ OK ]
Shutting down sm-client: [ OK ]
Shutting down sendmail: [ OK ]
Stopping crond: [ OK ]
Shutting down ntpd: [ OK ]
Stopping block device availability: Deactivating block devices:
[MD]: deactivating raid0 device md0... skipping
[LVM]: deactivating Volume Group docker... done
[ OK ]
Stopping auditd: [395901.470975] audit: type=1305 audit(1495033354.081:1097): audit_pid=0 old=109278 auid=500 ses=91 res=1
[ OK ]
[395901.565585] audit: type=1305 audit(1495033354.177:1098): audit_enabled=0 old=1 auid=4294967295 ses=4294967295 res=1
Shutting down system logger: [ OK ]
Shutting down interface eth0: [ OK ]
Shutting down loopback interface: [ OK ]
Stopping cgconfig service: [ OK ]
Stopping monitoring for VG docker: WARNING: Failed to connect to lvmetad. Falling back to device scanning.
WARNING: Failed to connect to lvmetad. Falling back to device scanning.
[ OK ]
Stopping rngd: [ OK ]
Sending all processes the TERM signal... init: ecs post-stop process (114746) killed by TERM signal
[ OK ]
Sending all processes the KILL signal... [ OK ]
Saving random seed: [ OK ]
Unmounting file systems: [ OK ]
init: Re-executing /sbin/init
[395907.704988] EXT4-fs (xvda1): re-mounted. Opts: (null)
Please stand by while rebooting the system...
[395908.933545] xenbus: xenbus_dev_shutdown: device/pci/0: Initialising != Connected, skipping
[395908.938776] reboot: Restarting system
[395908.941265] reboot: machine restart
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment