node-a.log

solr_gc_log_20151102_2133:2015-11-02T21:23:00.782+0100: 50711.219: Total time for which application threads were stopped: 0.0032601 seconds, Stopping threads took: 0.0021462 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:01.312+0100: 50711.748: Total time for which application threads were stopped: 0.0750899 seconds, Stopping threads took: 0.0002942 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:01.347+0100: 50711.784: Total time for which application threads were stopped: 0.0016255 seconds, Stopping threads took: 0.0006437 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:05.393+0100: 50715.829: Total time for which application threads were stopped: 0.0013802 seconds, Stopping threads took: 0.0002940 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:05.622+0100: 50716.058: Total time for which application threads were stopped: 0.0017050 seconds, Stopping threads took: 0.0004218 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:06.111+0100: 50716.547: Total time for which application threads were stopped: 0.0023933 seconds, Stopping threads took: 0.0009414 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:06.289+0100: 50716.725: Total time for which application threads were stopped: 0.0048633 seconds, Stopping threads took: 0.0033586 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:07.569+0100: 50718.006: Total time for which application threads were stopped: 0.0030649 seconds, Stopping threads took: 0.0002601 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:08.303+0100: 50718.740: Total time for which application threads were stopped: 0.2711953 seconds, Stopping threads took: 0.0006038 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:13.928+0100: 50724.364: Total time for which application threads were stopped: 1.9690664 seconds, Stopping threads took: 0.0004015 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:20.002+0100: 50730.438: Total time for which application threads were stopped: 2.3218528 seconds, Stopping threads took: 0.0002353 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:20.130+0100: 50730.566: Total time for which application threads were stopped: 0.0830621 seconds, Stopping threads took: 0.0009832 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:21.511+0100: 50731.947: Total time for which application threads were stopped: 0.0128296 seconds, Stopping threads took: 0.0115313 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:22.749+0100: 50733.186: Total time for which application threads were stopped: 0.0014764 seconds, Stopping threads took: 0.0002882 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:27.636+0100: 50738.072: Total time for which application threads were stopped: 0.0018849 seconds, Stopping threads took: 0.0002786 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:30.983+0100: 50741.419: Total time for which application threads were stopped: 0.4868475 seconds, Stopping threads took: 0.0004130 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:33.574+0100: 50744.011: Total time for which application threads were stopped: 0.0052904 seconds, Stopping threads took: 0.0033387 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:33.578+0100: 50744.015: Total time for which application threads were stopped: 0.0013794 seconds, Stopping threads took: 0.0002703 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:34.044+0100: 50744.481: Total time for which application threads were stopped: 0.3988700 seconds, Stopping threads took: 0.0013893 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:35.698+0100: 50746.134: Total time for which application threads were stopped: 0.4078584 seconds, Stopping threads took: 0.0003149 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:35.780+0100: 50746.216: Total time for which application threads were stopped: 0.0563207 seconds, Stopping threads took: 0.0005403 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:36.027+0100: 50746.463: Total time for which application threads were stopped: 0.0020367 seconds, Stopping threads took: 0.0002674 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:36.329+0100: 50746.766: Total time for which application threads were stopped: 0.0035497 seconds, Stopping threads took: 0.0018170 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:36.394+0100: 50746.830: Total time for which application threads were stopped: 0.0054555 seconds, Stopping threads took: 0.0004214 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:36.430+0100: 50746.866: Total time for which application threads were stopped: 0.0032014 seconds, Stopping threads took: 0.0014797 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:36.434+0100: 50746.871: Total time for which application threads were stopped: 0.0041302 seconds, Stopping threads took: 0.0006809 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:36.439+0100: 50746.876: Total time for which application threads were stopped: 0.0015261 seconds, Stopping threads took: 0.0003974 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:36.471+0100: 50746.907: Total time for which application threads were stopped: 0.0019674 seconds, Stopping threads took: 0.0002603 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:37.459+0100: 50747.895: Total time for which application threads were stopped: 0.4779234 seconds, Stopping threads took: 0.0002497 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:38.462+0100: 50748.898: Total time for which application threads were stopped: 0.0030403 seconds, Stopping threads took: 0.0011067 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:39.027+0100: 50749.463: Total time for which application threads were stopped: 0.5487461 seconds, Stopping threads took: 0.0002629 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:39.032+0100: 50749.469: Total time for which application threads were stopped: 0.0044701 seconds, Stopping threads took: 0.0010154 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:40.048+0100: 50750.484: Total time for which application threads were stopped: 0.0032273 seconds, Stopping threads took: 0.0004158 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:41.050+0100: 50751.486: Total time for which application threads were stopped: 0.0020942 seconds, Stopping threads took: 0.0002718 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:41.900+0100: 50752.336: Total time for which application threads were stopped: 0.4297043 seconds, Stopping threads took: 0.0003049 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:42.906+0100: 50753.343: Total time for which application threads were stopped: 0.0049752 seconds, Stopping threads took: 0.0018383 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:44.100+0100: 50754.536: Total time for which application threads were stopped: 0.7684205 seconds, Stopping threads took: 0.0004825 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:45.104+0100: 50755.540: Total time for which application threads were stopped: 0.0038598 seconds, Stopping threads took: 0.0006001 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:47.008+0100: 50757.445: Total time for which application threads were stopped: 0.6883382 seconds, Stopping threads took: 0.0004436 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:48.016+0100: 50758.452: Total time for which application threads were stopped: 0.0072100 seconds, Stopping threads took: 0.0012430 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:49.487+0100: 50759.923: Total time for which application threads were stopped: 0.5314793 seconds, Stopping threads took: 0.0006864 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:51.715+0100: 50762.151: Total time for which application threads were stopped: 0.4131358 seconds, Stopping threads took: 0.0002821 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:52.046+0100: 50762.482: Total time for which application threads were stopped: 0.0024640 seconds, Stopping threads took: 0.0003465 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:52.088+0100: 50762.525: Total time for which application threads were stopped: 0.0058901 seconds, Stopping threads took: 0.0038801 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:52.152+0100: 50762.588: Total time for which application threads were stopped: 0.0027819 seconds, Stopping threads took: 0.0007734 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:52.244+0100: 50762.681: Total time for which application threads were stopped: 0.0060339 seconds, Stopping threads took: 0.0039646 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:52.311+0100: 50762.747: Total time for which application threads were stopped: 0.0118844 seconds, Stopping threads took: 0.0098283 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:53.128+0100: 50763.564: Total time for which application threads were stopped: 0.0128217 seconds, Stopping threads took: 0.0111028 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:53.196+0100: 50763.632: Total time for which application threads were stopped: 0.0045978 seconds, Stopping threads took: 0.0028819 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:54.259+0100: 50764.695: Total time for which application threads were stopped: 0.0018032 seconds, Stopping threads took: 0.0002728 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:55.828+0100: 50766.264: Total time for which application threads were stopped: 0.6104080 seconds, Stopping threads took: 0.0212270 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:55.840+0100: 50766.277: Total time for which application threads were stopped: 0.0072110 seconds, Stopping threads took: 0.0041471 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:55.842+0100: 50766.278: Total time for which application threads were stopped: 0.0015269 seconds, Stopping threads took: 0.0000560 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:55.844+0100: 50766.280: Total time for which application threads were stopped: 0.0016357 seconds, Stopping threads took: 0.0000521 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:55.846+0100: 50766.282: Total time for which application threads were stopped: 0.0018661 seconds, Stopping threads took: 0.0000524 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:55.847+0100: 50766.284: Total time for which application threads were stopped: 0.0017174 seconds, Stopping threads took: 0.0000694 seconds

node-b.log

solr_gc_log_20151102_2133:2015-11-02T21:23:02.689+0100: 50548.683: Total time for which application threads were stopped: 0.7491810 seconds, Stopping threads took: 0.7000004 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:05.350+0100: 50551.344: Total time for which application threads were stopped: 0.0017779 seconds, Stopping threads took: 0.0004965 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:05.352+0100: 50551.346: Total time for which application threads were stopped: 0.0010665 seconds, Stopping threads took: 0.0001823 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:06.892+0100: 50552.887: Total time for which application threads were stopped: 0.7031191 seconds, Stopping threads took: 0.7017255 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:07.703+0100: 50553.698: Total time for which application threads were stopped: 0.0015184 seconds, Stopping threads took: 0.0004256 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:10.279+0100: 50556.273: Total time for which application threads were stopped: 1.7104216 seconds, Stopping threads took: 0.0006879 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:12.530+0100: 50558.524: Total time for which application threads were stopped: 0.0025414 seconds, Stopping threads took: 0.0004911 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:12.532+0100: 50558.526: Total time for which application threads were stopped: 0.0018681 seconds, Stopping threads took: 0.0004247 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:12.534+0100: 50558.528: Total time for which application threads were stopped: 0.0015467 seconds, Stopping threads took: 0.0001304 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:19.987+0100: 50565.982: Total time for which application threads were stopped: 6.4482481 seconds, Stopping threads took: 0.0001836 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:20.154+0100: 50566.149: Total time for which application threads were stopped: 0.1659022 seconds, Stopping threads took: 0.0005357 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:20.950+0100: 50566.945: Total time for which application threads were stopped: 0.0074457 seconds, Stopping threads took: 0.0060178 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:21.208+0100: 50567.203: Total time for which application threads were stopped: 0.0028571 seconds, Stopping threads took: 0.0015520 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:30.420+0100: 50576.415: Total time for which application threads were stopped: 7.5185309 seconds, Stopping threads took: 0.0003392 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:30.863+0100: 50576.858: Total time for which application threads were stopped: 0.0167205 seconds, Stopping threads took: 0.0064834 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:31.626+0100: 50577.620: Total time for which application threads were stopped: 0.0177380 seconds, Stopping threads took: 0.0071897 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:31.665+0100: 50577.659: Total time for which application threads were stopped: 0.0108281 seconds, Stopping threads took: 0.0034037 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:31.696+0100: 50577.691: Total time for which application threads were stopped: 0.0127262 seconds, Stopping threads took: 0.0094410 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:31.706+0100: 50577.701: Total time for which application threads were stopped: 0.0046908 seconds, Stopping threads took: 0.0023709 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:31.739+0100: 50577.733: Total time for which application threads were stopped: 0.0068653 seconds, Stopping threads took: 0.0014699 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:31.765+0100: 50577.760: Total time for which application threads were stopped: 0.0083252 seconds, Stopping threads took: 0.0036949 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:31.865+0100: 50577.860: Total time for which application threads were stopped: 0.0751865 seconds, Stopping threads took: 0.0035848 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:31.907+0100: 50577.902: Total time for which application threads were stopped: 0.0066011 seconds, Stopping threads took: 0.0019313 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:31.951+0100: 50577.945: Total time for which application threads were stopped: 0.0283369 seconds, Stopping threads took: 0.0019091 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:32.003+0100: 50577.997: Total time for which application threads were stopped: 0.0038114 seconds, Stopping threads took: 0.0008928 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:32.006+0100: 50578.000: Total time for which application threads were stopped: 0.0023074 seconds, Stopping threads took: 0.0007250 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:35.031+0100: 50581.025: Total time for which application threads were stopped: 2.5183878 seconds, Stopping threads took: 0.0009327 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:37.489+0100: 50583.483: Total time for which application threads were stopped: 0.4506398 seconds, Stopping threads took: 0.4476472 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:37.567+0100: 50583.561: Total time for which application threads were stopped: 0.0114247 seconds, Stopping threads took: 0.0037569 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:37.607+0100: 50583.601: Total time for which application threads were stopped: 0.0057357 seconds, Stopping threads took: 0.0011079 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:39.214+0100: 50585.209: Total time for which application threads were stopped: 0.1250833 seconds, Stopping threads took: 0.0004985 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:45.559+0100: 50591.554: Total time for which application threads were stopped: 0.3412998 seconds, Stopping threads took: 0.0009240 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:47.996+0100: 50593.991: Total time for which application threads were stopped: 0.4370315 seconds, Stopping threads took: 0.4318732 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:50.282+0100: 50596.277: Total time for which application threads were stopped: 0.0061285 seconds, Stopping threads took: 0.0007104 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:50.287+0100: 50596.281: Total time for which application threads were stopped: 0.0044474 seconds, Stopping threads took: 0.0001501 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:50.291+0100: 50596.285: Total time for which application threads were stopped: 0.0036864 seconds, Stopping threads took: 0.0002302 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:52.022+0100: 50598.016: Total time for which application threads were stopped: 0.0229247 seconds, Stopping threads took: 0.0004033 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:52.050+0100: 50598.044: Total time for which application threads were stopped: 0.0061820 seconds, Stopping threads took: 0.0039628 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:52.099+0100: 50598.094: Total time for which application threads were stopped: 0.0044143 seconds, Stopping threads took: 0.0020220 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:52.767+0100: 50598.762: Total time for which application threads were stopped: 0.1532794 seconds, Stopping threads took: 0.0039637 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:53.306+0100: 50599.300: Total time for which application threads were stopped: 0.0070809 seconds, Stopping threads took: 0.0042573 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:53.308+0100: 50599.303: Total time for which application threads were stopped: 0.0017538 seconds, Stopping threads took: 0.0002613 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:58.610+0100: 50604.605: Total time for which application threads were stopped: 1.6016317 seconds, Stopping threads took: 1.2107958 seconds

node-c.log

solr_gc_log_20151102_2133:2015-11-02T21:23:00.185+0100: 197766.840: Total time for which application threads were stopped: 2.5812616 seconds, Stopping threads took: 0.0006437 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:00.442+0100: 197767.097: Total time for which application threads were stopped: 0.0032229 seconds, Stopping threads took: 0.0024375 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:00.664+0100: 197767.319: Total time for which application threads were stopped: 0.0058823 seconds, Stopping threads took: 0.0048641 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:00.672+0100: 197767.327: Total time for which application threads were stopped: 0.0063896 seconds, Stopping threads took: 0.0041063 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:03.523+0100: 197770.179: Total time for which application threads were stopped: 1.7039386 seconds, Stopping threads took: 0.0024372 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:07.399+0100: 197774.054: Total time for which application threads were stopped: 2.7678915 seconds, Stopping threads took: 0.0003005 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:07.467+0100: 197774.123: Total time for which application threads were stopped: 0.0021343 seconds, Stopping threads took: 0.0011017 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:07.521+0100: 197774.177: Total time for which application threads were stopped: 0.0082282 seconds, Stopping threads took: 0.0070830 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:07.961+0100: 197774.616: Total time for which application threads were stopped: 0.0019936 seconds, Stopping threads took: 0.0008072 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:08.066+0100: 197774.722: Total time for which application threads were stopped: 0.0064655 seconds, Stopping threads took: 0.0053215 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:14.129+0100: 197780.785: Total time for which application threads were stopped: 5.4362483 seconds, Stopping threads took: 0.0005824 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:14.309+0100: 197780.964: Total time for which application threads were stopped: 0.1219095 seconds, Stopping threads took: 0.0009303 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:14.767+0100: 197781.423: Total time for which application threads were stopped: 0.0058657 seconds, Stopping threads took: 0.0035613 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:14.770+0100: 197781.425: Total time for which application threads were stopped: 0.0020902 seconds, Stopping threads took: 0.0005383 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:14.775+0100: 197781.430: Total time for which application threads were stopped: 0.0032839 seconds, Stopping threads took: 0.0006313 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:18.934+0100: 197785.589: Total time for which application threads were stopped: 3.1924282 seconds, Stopping threads took: 0.0023592 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:19.684+0100: 197786.339: Total time for which application threads were stopped: 0.0092314 seconds, Stopping threads took: 0.0074251 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:42.092+0100: 197808.747: Total time for which application threads were stopped: 21.8814073 seconds, Stopping threads took: 0.0017399 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:43.024+0100: 197809.679: Total time for which application threads were stopped: 0.1272598 seconds, Stopping threads took: 0.0184684 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:43.050+0100: 197809.705: Total time for which application threads were stopped: 0.0253849 seconds, Stopping threads took: 0.0002253 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:43.169+0100: 197809.825: Total time for which application threads were stopped: 0.0186858 seconds, Stopping threads took: 0.0111397 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:43.346+0100: 197810.001: Total time for which application threads were stopped: 0.0363829 seconds, Stopping threads took: 0.0342449 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:43.372+0100: 197810.027: Total time for which application threads were stopped: 0.0066367 seconds, Stopping threads took: 0.0051758 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:43.396+0100: 197810.052: Total time for which application threads were stopped: 0.0111905 seconds, Stopping threads took: 0.0097474 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:43.399+0100: 197810.054: Total time for which application threads were stopped: 0.0011686 seconds, Stopping threads took: 0.0002453 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:43.412+0100: 197810.067: Total time for which application threads were stopped: 0.0046485 seconds, Stopping threads took: 0.0032254 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:43.426+0100: 197810.081: Total time for which application threads were stopped: 0.0047544 seconds, Stopping threads took: 0.0032332 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:43.446+0100: 197810.101: Total time for which application threads were stopped: 0.0150757 seconds, Stopping threads took: 0.0133430 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:43.464+0100: 197810.119: Total time for which application threads were stopped: 0.0154576 seconds, Stopping threads took: 0.0139096 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:43.466+0100: 197810.121: Total time for which application threads were stopped: 0.0013513 seconds, Stopping threads took: 0.0004039 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:43.492+0100: 197810.147: Total time for which application threads were stopped: 0.0074176 seconds, Stopping threads took: 0.0057115 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:43.511+0100: 197810.166: Total time for which application threads were stopped: 0.0139500 seconds, Stopping threads took: 0.0119959 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:43.544+0100: 197810.199: Total time for which application threads were stopped: 0.0111803 seconds, Stopping threads took: 0.0095931 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:43.546+0100: 197810.202: Total time for which application threads were stopped: 0.0015704 seconds, Stopping threads took: 0.0002618 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:43.559+0100: 197810.215: Total time for which application threads were stopped: 0.0028567 seconds, Stopping threads took: 0.0014592 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:43.575+0100: 197810.230: Total time for which application threads were stopped: 0.0087569 seconds, Stopping threads took: 0.0073201 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:43.577+0100: 197810.232: Total time for which application threads were stopped: 0.0014069 seconds, Stopping threads took: 0.0001850 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:43.586+0100: 197810.242: Total time for which application threads were stopped: 0.0048300 seconds, Stopping threads took: 0.0031240 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:43.621+0100: 197810.277: Total time for which application threads were stopped: 0.0135370 seconds, Stopping threads took: 0.0121349 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:43.636+0100: 197810.292: Total time for which application threads were stopped: 0.0077942 seconds, Stopping threads took: 0.0060352 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:43.657+0100: 197810.312: Total time for which application threads were stopped: 0.0100860 seconds, Stopping threads took: 0.0086391 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:43.669+0100: 197810.324: Total time for which application threads were stopped: 0.0070510 seconds, Stopping threads took: 0.0056540 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:44.679+0100: 197811.334: Total time for which application threads were stopped: 0.0093116 seconds, Stopping threads took: 0.0079796 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:49.066+0100: 197815.722: Total time for which application threads were stopped: 2.9025443 seconds, Stopping threads took: 0.0004030 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:53.330+0100: 197819.986: Total time for which application threads were stopped: 0.0044242 seconds, Stopping threads took: 0.0030903 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:53.345+0100: 197820.001: Total time for which application threads were stopped: 0.0078060 seconds, Stopping threads took: 0.0025704 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:53.372+0100: 197820.028: Total time for which application threads were stopped: 0.0031722 seconds, Stopping threads took: 0.0017762 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:53.374+0100: 197820.029: Total time for which application threads were stopped: 0.0009481 seconds, Stopping threads took: 0.0002899 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:53.386+0100: 197820.042: Total time for which application threads were stopped: 0.0056864 seconds, Stopping threads took: 0.0045031 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:53.400+0100: 197820.055: Total time for which application threads were stopped: 0.0037818 seconds, Stopping threads took: 0.0024807 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:53.402+0100: 197820.057: Total time for which application threads were stopped: 0.0011462 seconds, Stopping threads took: 0.0004157 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:53.405+0100: 197820.060: Total time for which application threads were stopped: 0.0012116 seconds, Stopping threads took: 0.0003355 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:53.408+0100: 197820.063: Total time for which application threads were stopped: 0.0011385 seconds, Stopping threads took: 0.0002626 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:53.410+0100: 197820.065: Total time for which application threads were stopped: 0.0011773 seconds, Stopping threads took: 0.0004198 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:53.412+0100: 197820.068: Total time for which application threads were stopped: 0.0010120 seconds, Stopping threads took: 0.0003011 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:53.467+0100: 197820.123: Total time for which application threads were stopped: 0.0016561 seconds, Stopping threads took: 0.0003932 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:53.468+0100: 197820.124: Total time for which application threads were stopped: 0.0007740 seconds, Stopping threads took: 0.0001215 seconds

node-d.log

solr_gc_log_20151102_2133:2015-11-02T21:23:00.239+0100: 50650.618: Total time for which application threads were stopped: 0.0019431 seconds, Stopping threads took: 0.0012355 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:03.250+0100: 50653.629: Total time for which application threads were stopped: 2.0162130 seconds, Stopping threads took: 0.0026670 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:08.497+0100: 50658.875: Total time for which application threads were stopped: 4.4958600 seconds, Stopping threads took: 0.0012418 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:08.665+0100: 50659.044: Total time for which application threads were stopped: 0.0172779 seconds, Stopping threads took: 0.0140298 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:08.700+0100: 50659.079: Total time for which application threads were stopped: 0.0187766 seconds, Stopping threads took: 0.0168967 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:09.235+0100: 50659.614: Total time for which application threads were stopped: 0.0056198 seconds, Stopping threads took: 0.0037123 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:09.418+0100: 50659.797: Total time for which application threads were stopped: 0.0127410 seconds, Stopping threads took: 0.0097458 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:11.596+0100: 50661.975: Total time for which application threads were stopped: 1.0701624 seconds, Stopping threads took: 0.0004837 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:14.051+0100: 50664.429: Total time for which application threads were stopped: 0.1999769 seconds, Stopping threads took: 0.0033307 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:16.989+0100: 50667.368: Total time for which application threads were stopped: 1.2867717 seconds, Stopping threads took: 0.0022915 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:17.037+0100: 50667.415: Total time for which application threads were stopped: 0.0026931 seconds, Stopping threads took: 0.0017017 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:20.936+0100: 50671.315: Total time for which application threads were stopped: 0.8538756 seconds, Stopping threads took: 0.0038722 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:33.364+0100: 50683.742: Total time for which application threads were stopped: 2.3251010 seconds, Stopping threads took: 0.0003498 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:33.634+0100: 50684.013: Total time for which application threads were stopped: 0.0294056 seconds, Stopping threads took: 0.0256656 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:35.607+0100: 50685.986: Total time for which application threads were stopped: 0.2302112 seconds, Stopping threads took: 0.0065989 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:36.620+0100: 50686.999: Total time for which application threads were stopped: 0.0095327 seconds, Stopping threads took: 0.0054414 seconds
solr_gc_log_20151102_2133:2015-11-02T21:23:52.119+0100: 50702.498: Total time for which application threads were stopped: 15.2071321 seconds, Stopping threads took: 0.0015391 seconds

zookeeper.log

015-11-02 21:23:54,000 [myid:94] - INFO  [SessionTracker:ZooKeeperServer@325] - Expiring session 0x250b794b7bb1f94, timeout of 15000ms exceeded
2015-11-02 21:23:54,001 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@476] - Processed session termination for sessionid: 0x250b794b7bb1f94
2015-11-02 21:23:54,004 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x5e50b794b7c820e1 type:delete cxid:0x172c zxid:0x160001aa06 txntype:-1 reqpath:n/a Error Path:/solrcloud/collections/fr-catalogue/leaders/shard1 Error:KeeperErrorCode = NoNode for /solrcloud/collections/fr-catalogue/leaders/shard1
2015-11-02 21:23:54,006 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x5e50b794b7c820e1 type:create cxid:0x172d zxid:0x160001aa07 txntype:-1 reqpath:n/a Error Path:/solrcloud/overseer Error:KeeperErrorCode = NodeExists for /solrcloud/overseer
2015-11-02 21:23:54,099 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x250b794b7bb23fb type:create cxid:0xd6a zxid:0x160001aa0b txntype:-1 reqpath:n/a Error Path:/solrcloud/collections/onsite-sitemap/leaders/shard1 Error:KeeperErrorCode = NodeExists for /solrcloud/collections/onsite-sitemap/leaders/shard1
2015-11-02 21:23:54,113 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x250b794b7bb23fb type:create cxid:0xd6f zxid:0x160001aa0c txntype:-1 reqpath:n/a Error Path:/solrcloud/collections/customer-addresses/leaders/shard1 Error:KeeperErrorCode = NodeExists for /solrcloud/collections/customer-addresses/leaders/shard1
2015-11-02 21:23:54,137 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x250b794b7bb23fb type:create cxid:0xd76 zxid:0x160001aa0f txntype:-1 reqpath:n/a Error Path:/solrcloud/collections/order-products/leaders/shard1 Error:KeeperErrorCode = NodeExists for /solrcloud/collections/order-products/leaders/shard1
2015-11-02 21:23:54,143 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x250b794b7bb23fb type:create cxid:0xd7f zxid:0x160001aa10 txntype:-1 reqpath:n/a Error Path:/solrcloud/collections/fr-catalogue/leaders/shard2 Error:KeeperErrorCode = NodeExists for /solrcloud/collections/fr-catalogue/leaders/shard2
2015-11-02 21:23:54,143 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x250b794b7bb23fb type:create cxid:0xd80 zxid:0x160001aa11 txntype:-1 reqpath:n/a Error Path:/solrcloud/collections/seo-page-information/leaders/shard1 Error:KeeperErrorCode = NodeExists for /solrcloud/collections/seo-page-information/leaders/shard1
2015-11-02 21:23:54,215 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x250b794b7bb23fb type:create cxid:0xd85 zxid:0x160001aa12 txntype:-1 reqpath:n/a Error Path:/solrcloud/collections/active-return-orders/leaders/shard1 Error:KeeperErrorCode = NodeExists for /solrcloud/collections/active-return-orders/leaders/shard1
2015-11-02 21:23:55,105 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x250b794b7bb23fb type:create cxid:0xd8a zxid:0x160001aa13 txntype:-1 reqpath:n/a Error Path:/solrcloud/collections/onsite-sitemap/leaders/shard1 Error:KeeperErrorCode = NodeExists for /solrcloud/collections/onsite-sitemap/leaders/shard1
2015-11-02 21:23:55,119 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x250b794b7bb23fb type:create cxid:0xd8f zxid:0x160001aa14 txntype:-1 reqpath:n/a Error Path:/solrcloud/collections/customer-addresses/leaders/shard1 Error:KeeperErrorCode = NodeExists for /solrcloud/collections/customer-addresses/leaders/shard1
2015-11-02 21:23:55,144 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x250b794b7bb23fb type:create cxid:0xd94 zxid:0x160001aa15 txntype:-1 reqpath:n/a Error Path:/solrcloud/collections/order-products/leaders/shard1 Error:KeeperErrorCode = NodeExists for /solrcloud/collections/order-products/leaders/shard1
2015-11-02 21:23:55,150 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x250b794b7bb23fb type:create cxid:0xd9d zxid:0x160001aa16 txntype:-1 reqpath:n/a Error Path:/solrcloud/collections/fr-catalogue/leaders/shard2 Error:KeeperErrorCode = NodeExists for /solrcloud/collections/fr-catalogue/leaders/shard2
2015-11-02 21:23:55,151 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x250b794b7bb23fb type:create cxid:0xd9e zxid:0x160001aa17 txntype:-1 reqpath:n/a Error Path:/solrcloud/collections/seo-page-information/leaders/shard1 Error:KeeperErrorCode = NodeExists for /solrcloud/collections/seo-page-information/leaders/shard1
2015-11-02 21:23:55,222 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x250b794b7bb23fb type:create cxid:0xda3 zxid:0x160001aa18 txntype:-1 reqpath:n/a Error Path:/solrcloud/collections/active-return-orders/leaders/shard1 Error:KeeperErrorCode = NodeExists for /solrcloud/collections/active-return-orders/leaders/shard1
2015-11-02 21:23:56,116 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x250b794b7bb23fb type:create cxid:0xda8 zxid:0x160001aa19 txntype:-1 reqpath:n/a Error Path:/solrcloud/collections/onsite-sitemap/leaders/shard1 Error:KeeperErrorCode = NodeExists for /solrcloud/collections/onsite-sitemap/leaders/shard1
2015-11-02 21:23:56,135 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x250b794b7bb23fb type:create cxid:0xdad zxid:0x160001aa1a txntype:-1 reqpath:n/a Error Path:/solrcloud/collections/customer-addresses/leaders/shard1 Error:KeeperErrorCode = NodeExists for /solrcloud/collections/customer-addresses/leaders/shard1
2015-11-02 21:23:56,150 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x250b794b7bb23fb type:create cxid:0xdb2 zxid:0x160001aa1b txntype:-1 reqpath:n/a Error Path:/solrcloud/collections/order-products/leaders/shard1 Error:KeeperErrorCode = NodeExists for /solrcloud/collections/order-products/leaders/shard1
2015-11-02 21:23:56,165 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x250b794b7bb23fb type:create cxid:0xdbb zxid:0x160001aa1c txntype:-1 reqpath:n/a Error Path:/solrcloud/collections/seo-page-information/leaders/shard1 Error:KeeperErrorCode = NodeExists for /solrcloud/collections/seo-page-information/leaders/shard1
2015-11-02 21:23:56,166 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x250b794b7bb23fb type:create cxid:0xdbc zxid:0x160001aa1d txntype:-1 reqpath:n/a Error Path:/solrcloud/collections/fr-catalogue/leaders/shard2 Error:KeeperErrorCode = NodeExists for /solrcloud/collections/fr-catalogue/leaders/shard2
2015-11-02 21:23:56,230 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x250b794b7bb23fb type:create cxid:0xdc1 zxid:0x160001aa1e txntype:-1 reqpath:n/a Error Path:/solrcloud/collections/active-return-orders/leaders/shard1 Error:KeeperErrorCode = NodeExists for /solrcloud/collections/active-return-orders/leaders/shard1
2015-11-02 21:23:56,520 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x5e50b794b7c820e1 type:create cxid:0x1748 zxid:0x160001aa20 txntype:-1 reqpath:n/a Error Path:/solrcloud/overseer Error:KeeperErrorCode = NodeExists for /solrcloud/overseer
2015-11-02 21:23:58,299 [myid:94] - WARN  [SyncThread:94:FileTxnLog@321] - fsync-ing the write ahead log in SyncThread:94 took 1245ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
2015-11-02 21:23:58,307 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@476] - Processed session termination for sessionid: 0x5a50b794c0d827bc
2015-11-02 21:23:58,405 [myid:94] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@197] - Accepted socket connection from /10.41.199.120:52287
2015-11-02 21:23:58,407 [myid:94] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@821] - Processing mntr command from /10.41.199.120:52287
2015-11-02 21:23:58,408 [myid:94] - INFO  [Thread-7171:NIOServerCnxn@1001] - Closed socket connection for client /10.41.199.120:52287 (no session established for client)
2015-11-02 21:23:58,626 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x250b794b7bb23fb type:create cxid:0xdc8 zxid:0x160001aa28 txntype:-1 reqpath:n/a Error Path:/solrcloud/collections/active-return-orders/leaders/shard1 Error:KeeperErrorCode = NodeExists for /solrcloud/collections/active-return-orders/leaders/shard1
2015-11-02 21:23:58,643 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x250b794b7bb23fb type:create cxid:0xddc zxid:0x160001aa29 txntype:-1 reqpath:n/a Error Path:/solrcloud/collections/onsite-sitemap/leaders/shard1 Error:KeeperErrorCode = NodeExists for /solrcloud/collections/onsite-sitemap/leaders/shard1
2015-11-02 21:23:58,643 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x250b794b7bb23fb type:create cxid:0xdde zxid:0x160001aa2a txntype:-1 reqpath:n/a Error Path:/solrcloud/collections/fr-catalogue/leaders/shard2 Error:KeeperErrorCode = NodeExists for /solrcloud/collections/fr-catalogue/leaders/shard2
2015-11-02 21:23:58,645 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x250b794b7bb23fb type:create cxid:0xddf zxid:0x160001aa2b txntype:-1 reqpath:n/a Error Path:/solrcloud/collections/order-products/leaders/shard1 Error:KeeperErrorCode = NodeExists for /solrcloud/collections/order-products/leaders/shard1
2015-11-02 21:23:58,645 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x250b794b7bb23fb type:create cxid:0xde0 zxid:0x160001aa2c txntype:-1 reqpath:n/a Error Path:/solrcloud/collections/customer-addresses/leaders/shard1 Error:KeeperErrorCode = NodeExists for /solrcloud/collections/customer-addresses/leaders/shard1
2015-11-02 21:23:58,651 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x250b794b7bb23fb type:create cxid:0xde1 zxid:0x160001aa2d txntype:-1 reqpath:n/a Error Path:/solrcloud/collections/seo-page-information/leaders/shard1 Error:KeeperErrorCode = NodeExists for /solrcloud/collections/seo-page-information/leaders/shard1
2015-11-02 21:23:58,666 [myid:94] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@197] - Accepted socket connection from /10.41.199.141:41342
2015-11-02 21:23:58,666 [myid:94] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:ZooKeeperServer@839] - Client attempting to establish new session at /10.41.199.141:41342
2015-11-02 21:23:58,669 [myid:94] - INFO  [CommitProcessor:94:ZooKeeperServer@595] - Established session 0x5e50b794b7c82761 with negotiated timeout 10000 for client /10.41.199.141:41342
2015-11-02 21:23:58,674 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@476] - Processed session termination for sessionid: 0x5e50b794b7c82761
2015-11-02 21:23:58,684 [myid:94] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1001] - Closed socket connection for client /10.41.199.141:41342 which had sessionid 0x5e50b794b7c82761
2015-11-02 21:23:59,276 [myid:94] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@197] - Accepted socket connection from /10.41.199.120:52296
2015-11-02 21:23:59,277 [myid:94] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@821] - Processing mntr command from /10.41.199.120:52296
2015-11-02 21:23:59,278 [myid:94] - INFO  [Thread-7172:NIOServerCnxn@1001] - Closed socket connection for client /10.41.199.120:52296 (no session established for client)
2015-11-02 21:23:59,286 [myid:94] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@197] - Accepted socket connection from /10.41.199.120:52298
2015-11-02 21:23:59,287 [myid:94] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@821] - Processing mntr command from /10.41.199.120:52298
2015-11-02 21:23:59,288 [myid:94] - INFO  [Thread-7173:NIOServerCnxn@1001] - Closed socket connection for client /10.41.199.120:52298 (no session established for client)
2015-11-02 21:23:59,641 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x250b794b7bb23fb type:create cxid:0xde6 zxid:0x160001aa30 txntype:-1 reqpath:n/a Error Path:/solrcloud/collections/active-return-orders/leaders/shard1 Error:KeeperErrorCode = NodeExists for /solrcloud/collections/active-return-orders/leaders/shard1
2015-11-02 21:23:59,648 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x250b794b7bb23fb type:delete cxid:0xde7 zxid:0x160001aa31 txntype:-1 reqpath:n/a Error Path:/solrcloud/collections/active-return-orders/leader_elect/shard1/election/166835035578246139-core_node2-n_0000000061 Error:KeeperErrorCode = NoNode for /solrcloud/collections/active-return-orders/leader_elect/shard1/election/166835035578246139-core_node2-n_0000000061
2015-11-02 21:23:59,656 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x250b794b7bb23fb type:create cxid:0xdfc zxid:0x160001aa33 txntype:-1 reqpath:n/a Error Path:/solrcloud/collections/onsite-sitemap/leaders/shard1 Error:KeeperErrorCode = NodeExists for /solrcloud/collections/onsite-sitemap/leaders/shard1
2015-11-02 21:23:59,656 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x250b794b7bb23fb type:create cxid:0xdfd zxid:0x160001aa34 txntype:-1 reqpath:n/a Error Path:/solrcloud/collections/order-products/leaders/shard1 Error:KeeperErrorCode = NodeExists for /solrcloud/collections/order-products/leaders/shard1
2015-11-02 21:23:59,657 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x250b794b7bb23fb type:create cxid:0xdfe zxid:0x160001aa35 txntype:-1 reqpath:n/a Error Path:/solrcloud/collections/customer-addresses/leaders/shard1 Error:KeeperErrorCode = NodeExists for /solrcloud/collections/customer-addresses/leaders/shard1
2015-11-02 21:23:59,657 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x250b794b7bb23fb type:create cxid:0xdff zxid:0x160001aa36 txntype:-1 reqpath:n/a Error Path:/solrcloud/collections/fr-catalogue/leaders/shard2 Error:KeeperErrorCode = NodeExists for /solrcloud/collections/fr-catalogue/leaders/shard2
2015-11-02 21:23:59,679 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x250b794b7bb23fb type:create cxid:0xe03 zxid:0x160001aa37 txntype:-1 reqpath:n/a Error Path:/solrcloud/collections/seo-page-information/leaders/shard1 Error:KeeperErrorCode = NodeExists for /solrcloud/collections/seo-page-information/leaders/shard1
2015-11-02 21:24:00,211 [myid:94] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@197] - Accepted socket connection from /10.41.199.120:52311
2015-11-02 21:24:00,212 [myid:94] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@821] - Processing mntr command from /10.41.199.120:52311
2015-11-02 21:24:00,214 [myid:94] - INFO  [Thread-7174:NIOServerCnxn@1001] - Closed socket connection for client /10.41.199.120:52311 (no session established for client)
2015-11-02 21:24:00,684 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x250b794b7bb23fb type:create cxid:0xe14 zxid:0x160001aa38 txntype:-1 reqpath:n/a Error Path:/solrcloud/collections/onsite-sitemap/leaders/shard1 Error:KeeperErrorCode = NodeExists for /solrcloud/collections/onsite-sitemap/leaders/shard1
2015-11-02 21:24:00,684 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x250b794b7bb23fb type:create cxid:0xe15 zxid:0x160001aa39 txntype:-1 reqpath:n/a Error Path:/solrcloud/collections/customer-addresses/leaders/shard1 Error:KeeperErrorCode = NodeExists for /solrcloud/collections/customer-addresses/leaders/shard1
2015-11-02 21:24:00,685 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x250b794b7bb23fb type:create cxid:0xe16 zxid:0x160001aa3a txntype:-1 reqpath:n/a Error Path:/solrcloud/collections/fr-catalogue/leaders/shard2 Error:KeeperErrorCode = NodeExists for /solrcloud/collections/fr-catalogue/leaders/shard2
2015-11-02 21:24:00,686 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x250b794b7bb23fb type:create cxid:0xe17 zxid:0x160001aa3b txntype:-1 reqpath:n/a Error Path:/solrcloud/collections/order-products/leaders/shard1 Error:KeeperErrorCode = NodeExists for /solrcloud/collections/order-products/leaders/shard1
2015-11-02 21:24:00,707 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x250b794b7bb23fb type:delete cxid:0xe1a zxid:0x160001aa3d txntype:-1 reqpath:n/a Error Path:/solrcloud/collections/customer-addresses/leader_elect/shard1/election/166835035578246139-core_node2-n_0000000061 Error:KeeperErrorCode = NoNode for /solrcloud/collections/customer-addresses/leader_elect/shard1/election/166835035578246139-core_node2-n_0000000061
2015-11-02 21:24:00,709 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x250b794b7bb23fb type:delete cxid:0xe1b zxid:0x160001aa3e txntype:-1 reqpath:n/a Error Path:/solrcloud/collections/fr-catalogue/leader_elect/shard2/election/166835035578246139-core_node2-n_0000000003 Error:KeeperErrorCode = NoNode for /solrcloud/collections/fr-catalogue/leader_elect/shard2/election/166835035578246139-core_node2-n_0000000003
2015-11-02 21:24:00,738 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x250b794b7bb23fb type:create cxid:0xe2a zxid:0x160001aa45 txntype:-1 reqpath:n/a Error Path:/solrcloud/collections/seo-page-information/leaders/shard1 Error:KeeperErrorCode = NodeExists for /solrcloud/collections/seo-page-information/leaders/shard1
2015-11-02 21:24:00,755 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x5a50b794c0d827bb type:create cxid:0x158 zxid:0x160001aa47 txntype:-1 reqpath:n/a Error Path:/solrcloud/overseer Error:KeeperErrorCode = NodeExists for /solrcloud/overseer
2015-11-02 21:24:00,755 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x5a50b794c0d827bb type:create cxid:0x159 zxid:0x160001aa48 txntype:-1 reqpath:n/a Error Path:/solrcloud/overseer Error:KeeperErrorCode = NodeExists for /solrcloud/overseer
2015-11-02 21:24:00,781 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x250b794b7bb23fb type:delete cxid:0xe2d zxid:0x160001aa4d txntype:-1 reqpath:n/a Error Path:/solrcloud/collections/seo-page-information/leader_elect/shard1/election/166835035578246139-core_node2-n_0000000061 Error:KeeperErrorCode = NoNode for /solrcloud/collections/seo-page-information/leader_elect/shard1/election/166835035578246139-core_node2-n_0000000061
2015-11-02 21:24:02,841 [myid:94] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxnFactory@197] - Accepted socket connection from /10.41.199.201:59856
2015-11-02 21:24:02,856 [myid:94] - WARN  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@349] - caught end of stream exception
2015-11-02 21:24:02,857 [myid:94] - INFO  [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@1001] - Closed socket connection for client /10.41.199.201:59856 (no session established for client)
2015-11-02 21:24:03,303 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x5a50b794c0d827bb type:create cxid:0x1ac zxid:0x160001aa6c txntype:-1 reqpath:n/a Error Path:/solrcloud/overseer Error:KeeperErrorCode = NodeExists for /solrcloud/overseer
2015-11-02 21:24:03,334 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x5a50b794c0d827bb type:create cxid:0x1b6 zxid:0x160001aa6f txntype:-1 reqpath:n/a Error Path:/solrcloud/overseer Error:KeeperErrorCode = NodeExists for /solrcloud/overseer
2015-11-02 21:24:05,696 [myid:94] - WARN  [SyncThread:94:FileTxnLog@321] - fsync-ing the write ahead log in SyncThread:94 took 1702ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
2015-11-02 21:24:17,938 [myid:94] - WARN  [SyncThread:94:FileTxnLog@321] - fsync-ing the write ahead log in SyncThread:94 took 1462ms which will adversely effect operation latency. See the ZooKeeper troubleshooting guide
2015-11-02 21:24:19,813 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x250b794b7bb25bb type:delete cxid:0x59 zxid:0x160001aabc txntype:-1 reqpath:n/a Error Path:/solrcloud/overseer_elect/election/166835035578245012-10.41.199.201:9004_solr-n_0000000185 Error:KeeperErrorCode = NoNode for /solrcloud/overseer_elect/election/166835035578245012-10.41.199.201:9004_solr-n_0000000185
2015-11-02 21:24:19,870 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x250b794b7bb25bb type:delete cxid:0x75 zxid:0x160001aabe txntype:-1 reqpath:n/a Error Path:/solrcloud/live_nodes/10.41.199.201:9004_solr Error:KeeperErrorCode = NoNode for /solrcloud/live_nodes/10.41.199.201:9004_solr
2015-11-02 21:24:19,897 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x250b794b7bb25bb type:delete cxid:0x7b zxid:0x160001aac0 txntype:-1 reqpath:n/a Error Path:/solrcloud/collections/suggestion/leader_elect/shard2/election/166835035578245012-core_node1-n_0000000118 Error:KeeperErrorCode = NoNode for /solrcloud/collections/suggestion/leader_elect/shard2/election/166835035578245012-core_node1-n_0000000118
2015-11-02 21:24:19,907 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x250b794b7bb25bb type:delete cxid:0x7c zxid:0x160001aac1 txntype:-1 reqpath:n/a Error Path:/solrcloud/collections/seo-page-information/leader_elect/shard2/election/166835035578245012-core_node1-n_0000000121 Error:KeeperErrorCode = NoNode for /solrcloud/collections/seo-page-information/leader_elect/shard2/election/166835035578245012-core_node1-n_0000000121
2015-11-02 21:24:19,919 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x250b794b7bb25bb type:delete cxid:0x7e zxid:0x160001aac2 txntype:-1 reqpath:n/a Error Path:/solrcloud/collections/active-return-orders/leader_elect/shard2/election/166835035578245012-core_node1-n_0000000121 Error:KeeperErrorCode = NoNode for /solrcloud/collections/active-return-orders/leader_elect/shard2/election/166835035578245012-core_node1-n_0000000121
2015-11-02 21:24:19,923 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x250b794b7bb25bb type:delete cxid:0x7f zxid:0x160001aac3 txntype:-1 reqpath:n/a Error Path:/solrcloud/collections/static-content/leader_elect/shard1/election/166835035578245012-core_node1-n_0000000166 Error:KeeperErrorCode = NoNode for /solrcloud/collections/static-content/leader_elect/shard1/election/166835035578245012-core_node1-n_0000000166
2015-11-02 21:24:19,926 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x250b794b7bb25bb type:delete cxid:0x82 zxid:0x160001aac4 txntype:-1 reqpath:n/a Error Path:/solrcloud/collections/categories/leader_elect/shard2/election/166835035578245012-core_node1-n_0000000123 Error:KeeperErrorCode = NoNode for /solrcloud/collections/categories/leader_elect/shard2/election/166835035578245012-core_node1-n_0000000123
2015-11-02 21:24:19,929 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x250b794b7bb25bb type:delete cxid:0x83 zxid:0x160001aac5 txntype:-1 reqpath:n/a Error Path:/solrcloud/collections/customer-addresses/leader_elect/shard2/election/166835035578245012-core_node1-n_0000000121 Error:KeeperErrorCode = NoNode for /solrcloud/collections/customer-addresses/leader_elect/shard2/election/166835035578245012-core_node1-n_0000000121
2015-11-02 21:24:19,931 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x250b794b7bb25bb type:delete cxid:0x84 zxid:0x160001aac6 txntype:-1 reqpath:n/a Error Path:/solrcloud/collections/customers/leader_elect/shard2/election/166835035578245012-core_node1-n_0000000121 Error:KeeperErrorCode = NoNode for /solrcloud/collections/customers/leader_elect/shard2/election/166835035578245012-core_node1-n_0000000121
2015-11-02 21:24:19,943 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x250b794b7bb25bb type:delete cxid:0x8c zxid:0x160001aac8 txntype:-1 reqpath:n/a Error Path:/solrcloud/collections/order-products/leader_elect/shard2/election/166835035578245012-core_node1-n_0000000118 Error:KeeperErrorCode = NoNode for /solrcloud/collections/order-products/leader_elect/shard2/election/166835035578245012-core_node1-n_0000000118
2015-11-02 21:24:19,943 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x250b794b7bb25bb type:delete cxid:0x8d zxid:0x160001aac9 txntype:-1 reqpath:n/a Error Path:/solrcloud/collections/onsite-sitemap/leader_elect/shard2/election/166835035578245012-core_node1-n_0000000119 Error:KeeperErrorCode = NoNode for /solrcloud/collections/onsite-sitemap/leader_elect/shard2/election/166835035578245012-core_node1-n_0000000119
2015-11-02 21:24:19,958 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x250b794b7bb25bb type:delete cxid:0x98 zxid:0x160001aacc txntype:-1 reqpath:n/a Error Path:/solrcloud/collections/fr-catalogue/leader_elect/shard1/election/166835035578245012-core_node1-n_0000000000 Error:KeeperErrorCode = NoNode for /solrcloud/collections/fr-catalogue/leader_elect/shard1/election/166835035578245012-core_node1-n_0000000000
2015-11-02 21:24:19,958 [myid:94] - INFO  [ProcessThread(sid:94 cport:-1)::PrepRequestProcessor@627] - Got user-level KeeperException when processing sessionid:0x250b794b7bb25bb type:delete cxid:0x99 zxid:0x160001aacd txntype:-1 reqpath:n/a Error Path:/solrcloud/collections/catalogue/leader_elect/shard2/election/166835035578245012-core_node1-n_0000000165 Error:KeeperErrorCode = NoNode for /solrcloud/collections/catalogue/leader_elect/shard2/election/166835035578245012-core_node1-n_0000000165