Skip to content

Instantly share code, notes, and snippets.

@jtescher
Created January 28, 2014 21:37
Show Gist options
  • Save jtescher/8677043 to your computer and use it in GitHub Desktop.
Save jtescher/8677043 to your computer and use it in GitHub Desktop.
Neo4j Recovery Problem
###### SERVER ID 123 (Slave) ######
2014-01-28 21:15:30.828+0000 DEBUG [o.n.c.p.a.m.ProposerState$2]: ProposerState: proposer-[promise]->proposer from:cluster://10.10.1.251:5001 instance:614 conversation-id:123/944#
2014-01-28 21:15:30.828+0000 DEBUG [o.n.c.p.a.m.AcceptorState$2]: AcceptorState: acceptor-[accept]->acceptor from:cluster://10.10.1.102:5001 instance:614 conversation-id:123/944#
2014-01-28 21:15:30.828+0000 DEBUG [o.n.c.p.a.m.ProposerState$2]: ProposerState: proposer-[accepted]->proposer from:cluster://10.10.1.102:5001 instance:614 conversation-id:123/944#
2014-01-28 21:15:30.828+0000 DEBUG [o.n.c.p.a.m.ProposerState$2]: ProposerState: proposer-[accepted]->proposer from:cluster://10.10.1.251:5001 instance:614 conversation-id:123/944#
2014-01-28 21:15:30.829+0000 DEBUG [o.n.c.p.a.m.LearnerState$2]: Learned and closed instance 614 from conversation 123/944# and the content was Change cluster config, elected:272 as coordinator
2014-01-28 21:15:30.829+0000 DEBUG [o.n.c.p.a.m.LearnerState$2]: LearnerState: learner-[learn]->learner instance:614 conversation-id:123/944#
2014-01-28 21:15:30.829+0000 DEBUG [o.n.c.p.a.m.AtomicBroadcastState$3]: AtomicBroadcastState: broadcasting-[broadcastResponse]->broadcasting conversation-id:123/944#
2014-01-28 21:15:30.829+0000 DEBUG [o.n.c.p.c.ClusterState$4]: ClusterState: entered-[configurationChanged]->entered conversation-id:123/944#
2014-01-28 21:15:30.829+0000 INFO [o.n.k.h.HighAvailabilityConsoleLogger]: Instance 272 was elected as coordinator
2014-01-28 21:15:30.829+0000 DEBUG [o.n.c.p.a.m.ProposerState$2]: ProposerState: proposer-[accepted]->proposer from:cluster://10.10.1.110:5001 instance:614 conversation-id:123/944#
2014-01-28 21:16:03.059+0000 INFO [o.n.c.p.h.HeartbeatContext]: 123(me) is now suspecting 272
2014-01-28 21:16:03.062+0000 INFO [o.n.c.p.h.HeartbeatContext]: 131 is now suspecting 272
2014-01-28 21:16:03.062+0000 WARN [o.n.c.p.e.ClusterLeaveReelectionListener]: instance 272 is being demoted since it failed
2014-01-28 21:16:03.062+0000 INFO [o.n.k.h.HighAvailabilityConsoleLogger]: Instance 272 has failed
2014-01-28 21:16:03.062+0000 DEBUG [o.n.c.p.e.ElectionState]: Starting election process for role coordinator
2014-01-28 21:16:03.062+0000 DEBUG [o.n.c.p.e.ElectionState$2]: ElectionState: election-[demote]->election conversation-id:123/945#
2014-01-28 21:16:03.063+0000 DEBUG [o.n.c.p.e.ElectionState$2]: ElectionState: election-[vote:coordinator]->election from:cluster://10.10.1.102:5001 conversation-id:123/945#
2014-01-28 21:16:03.063+0000 DEBUG [o.n.c.p.e.ElectionState$2]: ElectionState: election-[voted]->election from:cluster://10.10.1.102:5001 conversation-id:123/945#
2014-01-28 21:16:03.064+0000 WARN [o.n.c.p.e.ElectionState]: Election could not pick a winner
2014-01-28 21:16:03.064+0000 DEBUG [o.n.c.p.e.ElectionState$2]: ElectionState: election-[voted]->election from:cluster://10.10.1.110:5001 conversation-id:123/945#
2014-01-28 21:16:03.076+0000 INFO [o.n.c.p.h.HeartbeatContext]: Notifying listeners that instance 272 is alive
2014-01-28 21:16:03.076+0000 INFO [o.n.k.h.HighAvailabilityConsoleLogger]: Instance 272 is alive
2014-01-28 21:16:03.076+0000 DEBUG [o.n.c.p.e.ElectionState]: Starting election process for role coordinator
2014-01-28 21:16:03.076+0000 INFO [o.n.c.p.e.ElectionContext]: Doing elections for role coordinator
2014-01-28 21:16:03.076+0000 DEBUG [o.n.c.p.e.ElectionState$2]: ElectionState: election-[performRoleElections]->election conversation-id:123/946#
2014-01-28 21:16:03.077+0000 DEBUG [o.n.c.p.e.ElectionState$2]: ElectionState: election-[vote:coordinator]->election from:cluster://10.10.1.102:5001 conversation-id:123/946#
2014-01-28 21:16:03.078+0000 DEBUG [o.n.c.p.e.ElectionState$2]: ElectionState: election-[voted]->election from:cluster://10.10.1.102:5001 conversation-id:123/946#
2014-01-28 21:16:03.078+0000 INFO [o.n.c.p.h.HeartbeatContext]: 131 is no longer suspecting 272
2014-01-28 21:16:03.079+0000 DEBUG [o.n.c.p.e.ElectionState]: Election already in progress for role coordinator
2014-01-28 21:16:03.079+0000 DEBUG [o.n.c.p.e.ElectionState$2]: ElectionState: election-[performRoleElections]->election from:cluster://10.10.1.110:5001 conversation-id:131/317#
2014-01-28 21:16:03.080+0000 DEBUG [o.n.c.p.e.ElectionState$2]: ElectionState: election-[voted]->election from:cluster://10.10.1.251:5001 conversation-id:123/946#
2014-01-28 21:16:03.080+0000 DEBUG [o.n.c.p.e.ElectionContext$2]: Elections ended up with list [272:DefaultElectionCredentials[serverId=272, latestTxId=11333095, currentWinner=true]]
2014-01-28 21:16:03.080+0000 DEBUG [o.n.c.p.e.ElectionState]: Elected 272 as coordinator
2014-01-28 21:16:03.080+0000 DEBUG [o.n.c.p.e.ElectionState$2]: ElectionState: election-[voted]->election from:cluster://10.10.1.110:5001 conversation-id:123/946#
2014-01-28 21:16:03.080+0000 DEBUG [o.n.c.p.a.m.ProposerState$2]: ProposerState: proposer-[propose]->proposer instance:615 conversation-id:123/946#
2014-01-28 21:16:03.081+0000 DEBUG [o.n.c.p.a.m.AcceptorState$2]: AcceptorState: acceptor-[prepare]->acceptor from:cluster://10.10.1.102:5001 instance:615 conversation-id:123/946#
2014-01-28 21:16:03.081+0000 DEBUG [o.n.c.p.a.m.ProposerState$2]: ProposerState: proposer-[promise]->proposer from:cluster://10.10.1.102:5001 instance:615 conversation-id:123/946#
2014-01-28 21:16:03.082+0000 DEBUG [o.n.c.p.a.m.ProposerState$2]: ProposerState: proposer-[promise]->proposer from:cluster://10.10.1.251:5001 instance:615 conversation-id:123/946#
2014-01-28 21:16:03.083+0000 DEBUG [o.n.c.p.a.m.ProposerState$2]: ProposerState: proposer-[promise]->proposer from:cluster://10.10.1.110:5001 instance:615 conversation-id:123/946#
2014-01-28 21:16:03.083+0000 DEBUG [o.n.c.p.a.m.AcceptorState$2]: AcceptorState: acceptor-[accept]->acceptor from:cluster://10.10.1.102:5001 instance:615 conversation-id:123/946#
2014-01-28 21:16:03.083+0000 DEBUG [o.n.c.p.a.m.ProposerState$2]: ProposerState: proposer-[accepted]->proposer from:cluster://10.10.1.102:5001 instance:615 conversation-id:123/946#
2014-01-28 21:16:03.084+0000 DEBUG [o.n.c.p.a.m.ProposerState$2]: ProposerState: proposer-[accepted]->proposer from:cluster://10.10.1.110:5001 instance:615 conversation-id:123/946#
2014-01-28 21:16:03.084+0000 DEBUG [o.n.c.p.a.m.LearnerState$2]: Learned and closed instance 615 from conversation 123/946# and the content was Change cluster config, elected:272 as coordinator
2014-01-28 21:16:03.084+0000 DEBUG [o.n.c.p.a.m.LearnerState$2]: LearnerState: learner-[learn]->learner instance:615 conversation-id:123/946#
2014-01-28 21:16:03.084+0000 DEBUG [o.n.c.p.a.m.AtomicBroadcastState$3]: AtomicBroadcastState: broadcasting-[broadcastResponse]->broadcasting conversation-id:123/946#
2014-01-28 21:16:03.084+0000 DEBUG [o.n.c.p.c.ClusterState$4]: ClusterState: entered-[configurationChanged]->entered conversation-id:123/946#
2014-01-28 21:16:03.085+0000 INFO [o.n.k.h.HighAvailabilityConsoleLogger]: Instance 272 was elected as coordinator
2014-01-28 21:16:03.135+0000 DEBUG [o.n.c.p.a.m.ProposerState$2]: ProposerState: proposer-[accepted]->proposer from:cluster://10.10.1.251:5001 instance:615 conversation-id:123/946#
2014-01-28 21:16:14.445+0000 INFO [o.n.c.p.h.HeartbeatContext]: 272 is now suspecting 131
2014-01-28 21:16:18.085+0000 INFO [o.n.c.p.h.HeartbeatContext]: 272 is no longer suspecting 131
###### SERVER ID 131 (Slave) ######
2014-01-28 21:15:30.825+0000 INFO [o.n.c.p.h.HeartbeatContext]: 123 is no longer suspecting 272
2014-01-28 21:15:30.825+0000 DEBUG [o.n.c.p.e.ElectionState$2]: ElectionState: election-[vote:coordinator]->election from:cluster://10.10.1.102:5001 conversation-id:123/944#
2014-01-28 21:15:30.826+0000 INFO [o.n.c.p.h.HeartbeatContext]: 272 is no longer suspecting 123
2014-01-28 21:15:30.827+0000 DEBUG [o.n.c.p.a.m.AcceptorState$2]: AcceptorState: acceptor-[prepare]->acceptor from:cluster://10.10.1.102:5001 instance:614 conversation-id:123/944#
2014-01-28 21:15:30.829+0000 DEBUG [o.n.c.p.a.m.AcceptorState$2]: AcceptorState: acceptor-[accept]->acceptor from:cluster://10.10.1.102:5001 instance:614 conversation-id:123/944#
2014-01-28 21:15:30.831+0000 DEBUG [o.n.c.p.a.m.LearnerState$2]: Learned and closed instance 614 from conversation 123/944# and the content was Change cluster config, elected:272 as coordinator
2014-01-28 21:15:30.831+0000 DEBUG [o.n.c.p.a.m.LearnerState$2]: LearnerState: learner-[learn]->learner from:cluster://10.10.1.102:5001 instance:614 conversation-id:123/944#
2014-01-28 21:15:30.831+0000 DEBUG [o.n.c.p.a.m.AtomicBroadcastState$3]: AtomicBroadcastState: broadcasting-[broadcastResponse]->broadcasting conversation-id:123/944#
2014-01-28 21:15:30.832+0000 DEBUG [o.n.c.p.c.ClusterState$4]: ClusterState: entered-[configurationChanged]->entered conversation-id:123/944#
2014-01-28 21:15:30.832+0000 INFO [o.n.k.h.HighAvailabilityConsoleLogger]: Instance 272 was elected as coordinator
2014-01-28 21:16:03.061+0000 INFO [o.n.c.p.h.HeartbeatContext]: 131(me) is now suspecting 272
2014-01-28 21:16:03.063+0000 INFO [o.n.c.p.h.HeartbeatContext]: 123 is now suspecting 272
2014-01-28 21:16:03.063+0000 WARN [o.n.c.p.e.ClusterLeaveReelectionListener]: instance 272 is being demoted since it failed
2014-01-28 21:16:03.063+0000 INFO [o.n.k.h.HighAvailabilityConsoleLogger]: Instance 272 has failed
2014-01-28 21:16:03.064+0000 DEBUG [o.n.c.p.e.ElectionState$2]: ElectionState: election-[demote]->election conversation-id:131/316#
2014-01-28 21:16:03.064+0000 DEBUG [o.n.c.p.e.ElectionState$2]: ElectionState: election-[vote:coordinator]->election from:cluster://10.10.1.102:5001 conversation-id:123/945#
2014-01-28 21:16:03.077+0000 INFO [o.n.c.p.h.HeartbeatContext]: Notifying listeners that instance 272 is alive
2014-01-28 21:16:03.077+0000 INFO [o.n.k.h.HighAvailabilityConsoleLogger]: Instance 272 is alive
2014-01-28 21:16:03.078+0000 DEBUG [o.n.c.p.e.ElectionState$2]: ElectionState: election-[performRoleElections]->election conversation-id:131/317#
2014-01-28 21:16:03.080+0000 INFO [o.n.c.p.h.HeartbeatContext]: 123 is no longer suspecting 272
2014-01-28 21:16:03.080+0000 DEBUG [o.n.c.p.e.ElectionState$2]: ElectionState: election-[vote:coordinator]->election from:cluster://10.10.1.102:5001 conversation-id:123/946#
2014-01-28 21:16:03.083+0000 DEBUG [o.n.c.p.a.m.AcceptorState$2]: AcceptorState: acceptor-[prepare]->acceptor from:cluster://10.10.1.102:5001 instance:615 conversation-id:123/946#
2014-01-28 21:16:03.085+0000 DEBUG [o.n.c.p.a.m.AcceptorState$2]: AcceptorState: acceptor-[accept]->acceptor from:cluster://10.10.1.102:5001 instance:615 conversation-id:123/946#
2014-01-28 21:16:03.087+0000 DEBUG [o.n.c.p.a.m.LearnerState$2]: Learned and closed instance 615 from conversation 123/946# and the content was Change cluster config, elected:272 as coordinator
2014-01-28 21:16:03.087+0000 DEBUG [o.n.c.p.a.m.LearnerState$2]: LearnerState: learner-[learn]->learner from:cluster://10.10.1.102:5001 instance:615 conversation-id:123/946#
2014-01-28 21:16:03.087+0000 DEBUG [o.n.c.p.a.m.AtomicBroadcastState$3]: AtomicBroadcastState: broadcasting-[broadcastResponse]->broadcasting conversation-id:123/946#
2014-01-28 21:16:03.087+0000 DEBUG [o.n.c.p.c.ClusterState$4]: ClusterState: entered-[configurationChanged]->entered conversation-id:123/946#
2014-01-28 21:16:03.087+0000 INFO [o.n.k.h.HighAvailabilityConsoleLogger]: Instance 272 was elected as coordinator
2014-01-28 21:16:14.447+0000 INFO [o.n.c.p.h.HeartbeatContext]: 272 is now suspecting 123
2014-01-28 21:16:18.094+0000 INFO [o.n.c.p.h.HeartbeatContext]: 272 is no longer suspecting 123
2014-01-28 21:17:07.852+0000 INFO [o.n.c.p.h.HeartbeatContext]: 272 is now suspecting 123
2014-01-28 21:17:08.134+0000 INFO [o.n.c.p.h.HeartbeatContext]: 272 is no longer suspecting 123
2014-01-28 21:18:03.843+0000 INFO [o.n.c.p.h.HeartbeatContext]: 123 is now suspecting 272
2014-01-28 21:18:03.847+0000 INFO [o.n.c.p.h.HeartbeatContext]: 131(me) is now suspecting 272
2014-01-28 21:18:03.847+0000 INFO [o.n.c.p.h.HeartbeatContext]: Notifying listeners that instance 272 is failed
2014-01-28 21:18:03.848+0000 WARN [o.n.c.p.e.ClusterLeaveReelectionListener]: instance 272 is being demoted since it failed
2014-01-28 21:18:03.848+0000 INFO [o.n.k.h.HighAvailabilityConsoleLogger]: Instance 272 has failed
2014-01-28 21:18:03.849+0000 DEBUG [o.n.c.p.e.ElectionState$2]: ElectionState: election-[demote]->election conversation-id:131/318#
2014-01-28 21:18:03.851+0000 DEBUG [o.n.c.p.e.ElectionState$2]: ElectionState: election-[vote:coordinator]->election from:cluster://10.10.1.102:5001 conversation-id:123/947#
2014-01-28 21:18:04.159+0000 INFO [o.n.c.p.h.HeartbeatContext]: Notifying listeners that instance 272 is alive
2014-01-28 21:18:04.159+0000 INFO [o.n.k.h.HighAvailabilityConsoleLogger]: Instance 272 is alive
2014-01-28 21:18:04.160+0000 DEBUG [o.n.c.p.e.ElectionState$2]: ElectionState: election-[performRoleElections]->election conversation-id:131/319#
2014-01-28 21:18:04.162+0000 INFO [o.n.c.p.h.HeartbeatContext]: 123 is no longer suspecting 272
2014-01-28 21:18:04.162+0000 DEBUG [o.n.c.p.e.ElectionState$2]: ElectionState: election-[vote:coordinator]->election from:cluster://10.10.1.102:5001 conversation-id:123/948#
2014-01-28 21:18:04.183+0000 INFO [o.n.c.p.h.HeartbeatContext]: 272 is now suspecting 123
2014-01-28 21:18:04.185+0000 DEBUG [o.n.c.p.a.m.AcceptorState$2]: AcceptorState: acceptor-[prepare]->acceptor from:cluster://10.10.1.102:5001 instance:616 conversation-id:123/948#
2014-01-28 21:18:04.187+0000 DEBUG [o.n.c.p.a.m.AcceptorState$2]: AcceptorState: acceptor-[accept]->acceptor from:cluster://10.10.1.102:5001 instance:616 conversation-id:123/948#
2014-01-28 21:18:04.190+0000 DEBUG [o.n.c.p.a.m.LearnerState$2]: Learned and closed instance 616 from conversation 123/948# and the content was Change cluster config, elected:272 as coordinator
2014-01-28 21:18:04.190+0000 DEBUG [o.n.c.p.a.m.LearnerState$2]: LearnerState: learner-[learn]->learner from:cluster://10.10.1.102:5001 instance:616 conversation-id:123/948#
2014-01-28 21:18:04.190+0000 DEBUG [o.n.c.p.a.m.AtomicBroadcastState$3]: AtomicBroadcastState: broadcasting-[broadcastResponse]->broadcasting conversation-id:123/948#
2014-01-28 21:18:04.190+0000 DEBUG [o.n.c.p.c.ClusterState$4]: ClusterState: entered-[configurationChanged]->entered conversation-id:123/948#
2014-01-28 21:18:04.191+0000 INFO [o.n.k.h.HighAvailabilityConsoleLogger]: Instance 272 was elected as coordinator
2014-01-28 21:18:04.256+0000 INFO [o.n.c.p.h.HeartbeatContext]: 272 is no longer suspecting 123
###### SERVER ID 272 (Master) ######
2014-01-28 21:18:25.867+0000 DEBUG [o.n.k.h.HighlyAvailableGraphDatabase]: Transaction 11333139 for nioneodb couldn't commit on enough slaves, desired 1, but could only commit at 0
2014-01-28 21:18:25.871+0000 DEBUG [o.n.k.h.HighlyAvailableGraphDatabase]: Transaction 6335976 for lucene-index couldn't commit on enough slaves, desired 1, but could only commit at 0
2014-01-28 21:18:25.875+0000 DEBUG [o.n.k.h.HighlyAvailableGraphDatabase]: Transaction 11333140 for nioneodb couldn't commit on enough slaves, desired 1, but could only commit at 0
2014-01-28 21:18:25.882+0000 DEBUG [o.n.k.h.HighlyAvailableGraphDatabase]: Transaction 11333141 for nioneodb couldn't commit on enough slaves, desired 1, but could only commit at 0
2014-01-28 21:18:25.892+0000 DEBUG [o.n.k.h.HighlyAvailableGraphDatabase]: Transaction 11333142 for nioneodb couldn't commit on enough slaves, desired 1, but could only commit at 0
2014-01-28 21:18:25.910+0000 DEBUG [o.n.k.h.HighlyAvailableGraphDatabase]: Transaction 11333143 for nioneodb couldn't commit on enough slaves, desired 1, but could only commit at 0
2014-01-28 21:18:25.912+0000 DEBUG [o.n.k.h.HighlyAvailableGraphDatabase]: Transaction 6335977 for lucene-index couldn't commit on enough slaves, desired 1, but could only commit at 0
2014-01-28 21:18:25.935+0000 DEBUG [o.n.k.h.HighlyAvailableGraphDatabase]: Transaction 6335978 for lucene-index couldn't commit on enough slaves, desired 1, but could only commit at 0
2014-01-28 21:18:25.977+0000 DEBUG [o.n.k.h.HighlyAvailableGraphDatabase]: Transaction 11333144 for nioneodb couldn't commit on enough slaves, desired 1, but could only commit at 0
2014-01-28 21:18:25.979+0000 DEBUG [o.n.k.h.HighlyAvailableGraphDatabase]: Transaction 6335979 for lucene-index couldn't commit on enough slaves, desired 1, but could only commit at 0
2014-01-28 21:18:25.994+0000 DEBUG [o.n.k.h.HighlyAvailableGraphDatabase]: Transaction 11333145 for nioneodb couldn't commit on enough slaves, desired 1, but could only commit at 0
2014-01-28 21:18:26.031+0000 DEBUG [o.n.k.h.HighlyAvailableGraphDatabase]: Transaction 11333146 for nioneodb couldn't commit on enough slaves, desired 1, but could only commit at 0
2014-01-28 21:18:26.034+0000 DEBUG [o.n.k.h.HighlyAvailableGraphDatabase]: Transaction 6335980 for lucene-index couldn't commit on enough slaves, desired 1, but could only commit at 0
2014-01-28 21:18:26.049+0000 DEBUG [o.n.k.h.HighlyAvailableGraphDatabase]: Transaction 11333147 for nioneodb couldn't commit on enough slaves, desired 1, but could only commit at 0
2014-01-28 21:18:26.080+0000 DEBUG [o.n.k.h.HighlyAvailableGraphDatabase]: Transaction 11333148 for nioneodb couldn't commit on enough slaves, desired 1, but could only commit at 0
2014-01-28 21:18:26.083+0000 DEBUG [o.n.k.h.HighlyAvailableGraphDatabase]: Transaction 6335981 for lucene-index couldn't commit on enough slaves, desired 1, but could only commit at 0
2014-01-28 21:18:36.377+0000 DEBUG [o.n.k.h.HighlyAvailableGraphDatabase]: GC Monitor: Application threads blocked for an additional 10214ms [total block time: 2988.42s]
2014-01-28 21:18:36.413+0000 DEBUG [o.n.k.h.HighlyAvailableGraphDatabase]: Transaction 6335982 for lucene-index couldn't commit on enough slaves, desired 1, but could only commit at 0
2014-01-28 21:18:36.530+0000 DEBUG [o.n.k.h.HighlyAvailableGraphDatabase]: Transaction 11333149 for nioneodb couldn't commit on enough slaves, desired 1, but could only commit at 0
2014-01-28 21:18:36.532+0000 DEBUG [o.n.k.h.HighlyAvailableGraphDatabase]: Transaction 6335983 for lucene-index couldn't commit on enough slaves, desired 1, but could only commit at 0
2014-01-28 21:18:36.562+0000 DEBUG [o.n.k.h.HighlyAvailableGraphDatabase]: Transaction 11333150 for nioneodb couldn't commit on enough slaves, desired 1, but could only commit at 0
2014-01-28 21:18:36.606+0000 DEBUG [o.n.k.h.HighlyAvailableGraphDatabase]: Transaction 11333151 for nioneodb couldn't commit on enough slaves, desired 1, but could only commit at 0
2014-01-28 21:18:36.609+0000 DEBUG [o.n.k.h.HighlyAvailableGraphDatabase]: Transaction 6335984 for lucene-index couldn't commit on enough slaves, desired 1, but could only commit at 0
2014-01-28 21:18:36.623+0000 DEBUG [o.n.k.h.HighlyAvailableGraphDatabase]: Transaction 11333152 for nioneodb couldn't commit on enough slaves, desired 1, but could only commit at 0
2014-01-28 21:18:36.664+0000 DEBUG [o.n.k.h.HighlyAvailableGraphDatabase]: Transaction 11333153 for nioneodb couldn't commit on enough slaves, desired 1, but could only commit at 0
2014-01-28 21:18:36.667+0000 DEBUG [o.n.k.h.HighlyAvailableGraphDatabase]: Transaction 6335985 for lucene-index couldn't commit on enough slaves, desired 1, but could only commit at 0
2014-01-28 21:18:46.986+0000 DEBUG [o.n.k.h.HighlyAvailableGraphDatabase]: GC Monitor: Application threads blocked for an additional 10203ms [total block time: 2998.623s]
2014-01-28 21:18:46.988+0000 DEBUG [o.n.k.h.HighlyAvailableGraphDatabase]: Transaction 6335986 for lucene-index couldn't commit on enough slaves, desired 1, but could only commit at 0
2014-01-28 21:18:47.030+0000 DEBUG [o.n.k.h.HighlyAvailableGraphDatabase]: Transaction 11333154 for nioneodb couldn't commit on enough slaves, desired 1, but could only commit at 0
2014-01-28 21:18:47.033+0000 DEBUG [o.n.k.h.HighlyAvailableGraphDatabase]: Transaction 6335987 for lucene-index couldn't commit on enough slaves, desired 1, but could only commit at 0
2014-01-28 21:18:47.076+0000 DEBUG [o.n.k.h.HighlyAvailableGraphDatabase]: Transaction 11333155 for nioneodb couldn't commit on enough slaves, desired 1, but could only commit at 0
2014-01-28 21:18:47.114+0000 DEBUG [o.n.k.h.HighlyAvailableGraphDatabase]: Transaction 11333156 for nioneodb couldn't commit on enough slaves, desired 1, but could only commit at 0
2014-01-28 21:18:47.120+0000 DEBUG [o.n.k.h.HighlyAvailableGraphDatabase]: Transaction 6335988 for lucene-index couldn't commit on enough slaves, desired 1, but could only commit at 0
2014-01-28 21:18:47.150+0000 DEBUG [o.n.k.h.HighlyAvailableGraphDatabase]: Transaction 11333157 for nioneodb couldn't commit on enough slaves, desired 1, but could only commit at 0
2014-01-28 21:18:47.192+0000 DEBUG [o.n.k.h.HighlyAvailableGraphDatabase]: guard-timeout:30000(+12711)ms
2014-01-28 21:18:47.216+0000 DEBUG [o.n.k.h.HighlyAvailableGraphDatabase]: Transaction 11333158 for nioneodb couldn't commit on enough slaves, desired 1, but could only commit at 0
2014-01-28 21:18:47.219+0000 DEBUG [o.n.k.h.HighlyAvailableGraphDatabase]: Transaction 6335989 for lucene-index couldn't commit on enough slaves, desired 1, but could only commit at 0
2014-01-28 21:18:53.456+0000 DEBUG [o.n.k.h.HighlyAvailableGraphDatabase]: GC Monitor: Application threads blocked for an additional 6169ms [total block time: 3004.792s]
2014-01-28 21:18:53.562+0000 DEBUG [o.n.k.h.HighlyAvailableGraphDatabase]: Transaction 6335990 for lucene-index couldn't commit on enough slaves, desired 1, but could only commit at 0
2014-01-28 21:18:53.599+0000 DEBUG [o.n.k.h.HighlyAvailableGraphDatabase]: Transaction 11333159 for nioneodb couldn't commit on enough slaves, desired 1, but could only commit at 0
2014-01-28 21:18:53.601+0000 DEBUG [o.n.k.h.HighlyAvailableGraphDatabase]: Transaction 6335991 for lucene-index couldn't commit on enough slaves, desired 1, but could only commit at 0
2014-01-28 21:18:53.608+0000 DEBUG [o.n.k.h.HighlyAvailableGraphDatabase]: Transaction 11333160 for nioneodb couldn't commit on enough slaves, desired 1, but could only commit at 0
2014-01-28 21:18:53.635+0000 DEBUG [o.n.k.h.HighlyAvailableGraphDatabase]: Transaction 11333161 for nioneodb couldn't commit on enough slaves, desired 1, but could only commit at 0
2014-01-28 21:18:53.638+0000 DEBUG [o.n.k.h.HighlyAvailableGraphDatabase]: Transaction 6335992 for lucene-index couldn't commit on enough slaves, desired 1, but could only commit at 0
2014-01-28 21:18:53.646+0000 DEBUG [o.n.k.h.HighlyAvailableGraphDatabase]: Transaction 11333162 for nioneodb couldn't commit on enough slaves, desired 1, but could only commit at 0
2014-01-28 21:18:53.662+0000 DEBUG [o.n.k.h.HighlyAvailableGraphDatabase]: Transaction 11333163 for nioneodb couldn't commit on enough slaves, desired 1, but could only commit at 0
2014-01-28 21:18:53.666+0000 DEBUG [o.n.k.h.HighlyAvailableGraphDatabase]: Transaction 6335993 for lucene-index couldn't commit on enough slaves, desired 1, but could only commit at 0
2014-01-28 21:18:53.720+0000 DEBUG [o.n.k.h.HighlyAvailableGraphDatabase]: Transaction 6335994 for lucene-index couldn't commit on enough slaves, desired 1, but could only commit at 0
2014-01-28 21:18:53.767+0000 DEBUG [o.n.k.h.HighlyAvailableGraphDatabase]: Transaction 11333164 for nioneodb couldn't commit on enough slaves, desired 1, but could only commit at 0
2014-01-28 21:18:53.770+0000 DEBUG [o.n.k.h.HighlyAvailableGraphDatabase]: Transaction 6335995 for lucene-index couldn't commit on enough slaves, desired 1, but could only commit at 0
2014-01-28 21:18:53.778+0000 DEBUG [o.n.k.h.HighlyAvailableGraphDatabase]: Transaction 11333165 for nioneodb couldn't commit on enough slaves, desired 1, but could only commit at 0
2014-01-28 21:18:53.816+0000 DEBUG [o.n.k.h.HighlyAvailableGraphDatabase]: Transaction 11333166 for nioneodb couldn't commit on enough slaves, desired 1, but could only commit at 0
2014-01-28 21:18:53.820+0000 DEBUG [o.n.k.h.HighlyAvailableGraphDatabase]: Transaction 6335996 for lucene-index couldn't commit on enough slaves, desired 1, but could only commit at 0
2014-01-28 21:18:53.845+0000 DEBUG [o.n.k.h.HighlyAvailableGraphDatabase]: Transaction 11333167 for nioneodb couldn't commit on enough slaves, desired 1, but could only commit at 0
2014-01-28 21:18:53.862+0000 DEBUG [o.n.k.h.HighlyAvailableGraphDatabase]: Transaction 11333168 for nioneodb couldn't commit on enough slaves, desired 1, but could only commit at 0
2014-01-28 21:18:53.866+0000 DEBUG [o.n.k.h.HighlyAvailableGraphDatabase]: Transaction 6335997 for lucene-index couldn't commit on enough slaves, desired 1, but could only commit at 0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment