-
-
Save aphyr/6402464 to your computer and use it in GitHub Desktop.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| In http://www.datastax.com/dev/blog/row-level-isolation and | |
| http://www.datastax.com/dev/blog/atomic-batches-in-cassandra-1-2, Datastax | |
| asserts that batch updates to multiple cells in a single row are | |
| guaranteed to be isolated from concurrent batches: | |
| "Cassandra 1.1 guarantees that if you update both the login and password in the | |
| same update (for the same row key) then no concurrent read may see only a | |
| partial update." | |
| "...the guarantee is that the new login cannot be persisted without the new | |
| password being persisted too (and vice-versa)." | |
| In this test, we have a simple table schema: each row consists of an ID, an A | |
| column, and a B column. Five concurrent clients write sequentially to a | |
| thousand rows. Each client generates a random identifier, and tries to write | |
| that identifier to both the A and B fields for a given row. For reasons that | |
| will become clear later, we write +identifier to A, and -identifier to B. If | |
| the isolation invariant holds, A should always be equal to -B: we shouldn't see | |
| A=1 and B=-4, for instance. | |
| 30 seconds into the run, we partition the Cassandra nodes into [n1 n2 | n3 n4 | |
| n5]. At 500 writes, we heal the network partition. The partition is not | |
| necessary to cause this problem, but it does improve the probabilities a bit. | |
| At the end of the run, we read back all the rows, and check to make sure that | |
| A=B in each one. If we ever see a conflict; e.g. the row update was not | |
| isolated, we consider that write "lost". In this case, almost every write which | |
| occurs with an identical timestamp to another will result in corruption. In | |
| this particular cluster and workload scheduler, the chances of 5 updates to a row | |
| occurring at the same timestamp is roughly 7.5%, but if we force identical | |
| timestamps or more contention, the corruption probability rises to 99% and | |
| higher. | |
| Why does this happen? It's because when Cassandra compares conflicting copies | |
| of a data, either via active anti-entropy (AAE), read repair, or hinted | |
| handoff, it resolves conflicts at the level of the *cell*, not the *row*. Let's | |
| say the conflict resolution system sees these two copies of a row: | |
| +----- Cell A ----------+----- Cell B -----------+ | |
| id | A value | A timestamp | B value | B timestamp | |
| 1 | 3 | 123 | -3 | 123 | |
| 1 | 5 | 123 | -5 | 123 | |
| There are three possible ways to resolve this conflict in a way which preserves | |
| row isolation: by selecting (A=3, B=-3), or (A=5, B=-5), or by throwing away | |
| A and B altogether. Instead, Cassandra looks at each *cell* in isolation. Its | |
| conflict resolution function chooses the copy with the higher timestamp, and | |
| if timestamps are identical, with the higher *value*. In cell A, both timestamps | |
| are 123, and the value 5 is greater than the value 3, so the value becomes 5. | |
| In cell B, -3 is greater than -5, so we preserve -3. The resulting row is then: | |
| id | A value | A timestamp | B value | B timestamp | |
| 1 | 5 | 123 | -3 | 123 | |
| which means row isolation is *not* preserved. This affects not only concurrent | |
| reads (if they happen to interact with a read-repair or other write propagation | |
| which causes conflict resolution), but *all* reads of the value permanently. | |
| This failure is also, as far as I can tell, independent of the inter-node | |
| consistency level. | |
| Here's a log of [write number, response, latency] for all 1000 writes: | |
| Run will take 100 seconds | |
| 0 :ok 11 ▏ | |
| 1 :ok 16 ▏ | |
| 2 :ok 21 ▏ | |
| 3 :ok 26 ▏ | |
| 4 :ok 25 ▏ | |
| 5 :ok 34 ▏ | |
| 6 :ok 38 ▏ | |
| 7 :ok 36 ▏ | |
| 8 :ok 30 ▏ | |
| 9 :ok 37 ▏ | |
| 10 :ok 26 ▏ | |
| 11 :ok 27 ▏ | |
| 12 :ok 23 ▏ | |
| 13 :ok 26 ▏ | |
| 14 :ok 31 ▏ | |
| 15 :ok 24 ▏ | |
| 16 :ok 27 ▏ | |
| 17 :ok 28 ▏ | |
| 18 :ok 29 ▏ | |
| 19 :ok 32 ▏ | |
| 20 :ok 29 ▏ | |
| 21 :ok 29 ▏ | |
| 22 :ok 33 ▏ | |
| 23 :ok 36 ▏ | |
| 24 :ok 36 ▏ | |
| 25 :ok 22 ▏ | |
| 26 :ok 26 ▏ | |
| 27 :ok 26 ▏ | |
| 28 :ok 20 ▏ | |
| 29 :ok 23 ▏ | |
| 30 :ok 17 ▏ | |
| 31 :ok 25 ▏ | |
| 32 :ok 27 ▏ | |
| 33 :ok 28 ▏ | |
| 34 :ok 30 ▏ | |
| 35 :ok 31 ▏ | |
| 36 :ok 30 ▏ | |
| 37 :ok 27 ▏ | |
| 38 :ok 32 ▏ | |
| 39 :ok 30 ▏ | |
| 40 :ok 59 ▏ | |
| 41 :ok 27 ▏ | |
| 42 :ok 61 ▏ | |
| 43 :ok 24 ▏ | |
| 44 :ok 63 ▎ | |
| 45 :ok 21 ▏ | |
| 46 :ok 26 ▏ | |
| 47 :ok 31 ▏ | |
| 48 :ok 29 ▏ | |
| 49 :ok 27 ▏ | |
| 50 :ok 30 ▏ | |
| 51 :ok 34 ▏ | |
| 52 :ok 35 ▏ | |
| 53 :ok 33 ▏ | |
| 54 :ok 30 ▏ | |
| 55 :ok 25 ▏ | |
| 56 :ok 28 ▏ | |
| 57 :ok 28 ▏ | |
| 58 :ok 25 ▏ | |
| 59 :ok 25 ▏ | |
| 60 :ok 29 ▏ | |
| 61 :ok 25 ▏ | |
| 62 :ok 27 ▏ | |
| 63 :ok 30 ▏ | |
| 64 :ok 24 ▏ | |
| 65 :ok 23 ▏ | |
| 66 :ok 30 ▏ | |
| 67 :ok 26 ▏ | |
| 68 :ok 29 ▏ | |
| 69 :ok 25 ▏ | |
| 70 :ok 23 ▏ | |
| 71 :ok 28 ▏ | |
| 72 :ok 18 ▏ | |
| 73 :ok 29 ▏ | |
| 74 :ok 33 ▏ | |
| 75 :ok 29 ▏ | |
| 76 :ok 28 ▏ | |
| 77 :ok 28 ▏ | |
| 78 :ok 25 ▏ | |
| 79 :ok 29 ▏ | |
| 80 :ok 33 ▏ | |
| 81 :ok 36 ▏ | |
| 82 :ok 29 ▏ | |
| 83 :ok 24 ▏ | |
| 84 :ok 37 ▏ | |
| 85 :ok 23 ▏ | |
| 86 :ok 25 ▏ | |
| 87 :ok 25 ▏ | |
| 88 :ok 18 ▏ | |
| 89 :ok 25 ▏ | |
| 90 :ok 21 ▏ | |
| 91 :ok 23 ▏ | |
| 92 :ok 29 ▏ | |
| 93 :ok 30 ▏ | |
| 94 :ok 28 ▏ | |
| 95 :ok 23 ▏ | |
| 96 :ok 27 ▏ | |
| 97 :ok 29 ▏ | |
| 98 :ok 27 ▏ | |
| 99 :ok 32 ▏ | |
| Initiating partition. | |
| 100 :ok 41 ▏ | |
| 101 :ok 26 ▏ | |
| 102 :ok 30 ▏ | |
| 103 :ok 41 ▏ | |
| 104 :ok 35 ▏ | |
| 105 :ok 20 ▏ | |
| 106 :ok 24 ▏ | |
| 107 :ok 23 ▏ | |
| 108 :ok 17 ▏ | |
| 109 :ok 24 ▏ | |
| Partitioned. | |
| 110 :ok 26 ▏ | |
| 111 :ok 26 ▏ | |
| 112 :ok 28 ▏ | |
| 113 :ok 27 ▏ | |
| 114 :ok 29 ▏ | |
| 115 :ok 16 ▏ | |
| 116 :ok 19 ▏ | |
| 117 :ok 18 ▏ | |
| 118 :error 10024 ████████████████████▏ | |
| Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| at com.datastax.driver.core.exceptions.WriteTimeoutException.copy (WriteTimeoutException.java:54) | |
| com.datastax.driver.core.ResultSetFuture.extractCauseFromExecutionException (ResultSetFuture.java:214) | |
| com.datastax.driver.core.ResultSetFuture.getUninterruptibly (ResultSetFuture.java:169) | |
| clojurewerkz.cassaforte.client$execute.doInvoke (client.clj:289) | |
| clojure.lang.RestFn.invoke (RestFn.java:457) | |
| jepsen.cassandra$isolation_app$reify__2559$fn__2560.invoke (cassandra.clj:186) | |
| jepsen.cassandra$isolation_app$reify__2559.add (cassandra.clj:177) | |
| jepsen.set_app$eval2420$fn__2443$G__2410__2446.invoke (set_app.clj:11) | |
| jepsen.set_app$eval2420$fn__2443$G__2409__2450.invoke (set_app.clj:11) | |
| clojure.lang.AFn.applyToHelper (AFn.java:163) | |
| clojure.lang.AFn.applyTo (AFn.java:151) | |
| clojure.core$apply.invoke (core.clj:619) | |
| clojure.core$partial$fn__4190.doInvoke (core.clj:2396) | |
| clojure.lang.RestFn.invoke (RestFn.java:408) | |
| jepsen.load$wrap_catch$catcher__2389.invoke (load.clj:110) | |
| jepsen.load$wrap_latency$measure_latency__2386.invoke (load.clj:100) | |
| jepsen.load$wrap_record_req$record_req__2398.invoke (load.clj:129) | |
| jepsen.console$wrap_ordered_log$logger__2346.invoke (console.clj:108) | |
| jepsen.load$map_fixed_rate$boss__2379$fn__2381.invoke (load.clj:65) | |
| clojure.lang.AFn.run (AFn.java:24) | |
| java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1145) | |
| java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:615) | |
| java.lang.Thread.run (Thread.java:722) | |
| Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| at com.datastax.driver.core.ResultSetFuture.convertException (ResultSetFuture.java:243) | |
| com.datastax.driver.core.ResultSetFuture$ResponseCallback.onSet (ResultSetFuture.java:119) | |
| com.datastax.driver.core.RequestHandler.setFinalResult (RequestHandler.java:202) | |
| com.datastax.driver.core.RequestHandler.onSet (RequestHandler.java:331) | |
| com.datastax.driver.core.Connection$Dispatcher.messageReceived (Connection.java:484) | |
| org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream (SimpleChannelUpstreamHandler.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:296) | |
| org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream (OneToOneDecoder.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream (OneToOneDecoder.java:68) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:296) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived (FrameDecoder.java:462) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode (FrameDecoder.java:443) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived (FrameDecoder.java:303) | |
| org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream (SimpleChannelUpstreamHandler.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:555) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:268) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:255) | |
| org.jboss.netty.channel.socket.nio.NioWorker.read (NioWorker.java:88) | |
| org.jboss.netty.channel.socket.nio.AbstractNioWorker.process (AbstractNioWorker.java:107) | |
| org.jboss.netty.channel.socket.nio.AbstractNioSelector.run (AbstractNioSelector.java:312) | |
| org.jboss.netty.channel.socket.nio.AbstractNioWorker.run (AbstractNioWorker.java:88) | |
| org.jboss.netty.channel.socket.nio.NioWorker.run (NioWorker.java:178) | |
| org.jboss.netty.util.ThreadRenamingRunnable.run (ThreadRenamingRunnable.java:108) | |
| org.jboss.netty.util.internal.DeadLockProofWorker$1.run (DeadLockProofWorker.java:42) | |
| java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1145) | |
| java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:615) | |
| java.lang.Thread.run (Thread.java:722) | |
| 119 :ok 20 ▏ | |
| 120 :ok 21 ▏ | |
| 121 :error 10024 ████████████████████▏ | |
| Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| at com.datastax.driver.core.exceptions.WriteTimeoutException.copy (WriteTimeoutException.java:54) | |
| com.datastax.driver.core.ResultSetFuture.extractCauseFromExecutionException (ResultSetFuture.java:214) | |
| com.datastax.driver.core.ResultSetFuture.getUninterruptibly (ResultSetFuture.java:169) | |
| clojurewerkz.cassaforte.client$execute.doInvoke (client.clj:289) | |
| clojure.lang.RestFn.invoke (RestFn.java:457) | |
| jepsen.cassandra$isolation_app$reify__2559$fn__2560.invoke (cassandra.clj:186) | |
| jepsen.cassandra$isolation_app$reify__2559.add (cassandra.clj:177) | |
| jepsen.set_app$eval2420$fn__2443$G__2410__2446.invoke (set_app.clj:11) | |
| jepsen.set_app$eval2420$fn__2443$G__2409__2450.invoke (set_app.clj:11) | |
| clojure.lang.AFn.applyToHelper (AFn.java:163) | |
| clojure.lang.AFn.applyTo (AFn.java:151) | |
| clojure.core$apply.invoke (core.clj:619) | |
| clojure.core$partial$fn__4190.doInvoke (core.clj:2396) | |
| clojure.lang.RestFn.invoke (RestFn.java:408) | |
| jepsen.load$wrap_catch$catcher__2389.invoke (load.clj:110) | |
| jepsen.load$wrap_latency$measure_latency__2386.invoke (load.clj:100) | |
| jepsen.load$wrap_record_req$record_req__2398.invoke (load.clj:129) | |
| jepsen.console$wrap_ordered_log$logger__2346.invoke (console.clj:108) | |
| jepsen.load$map_fixed_rate$boss__2379$fn__2381.invoke (load.clj:65) | |
| clojure.lang.AFn.run (AFn.java:24) | |
| java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1145) | |
| java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:615) | |
| java.lang.Thread.run (Thread.java:722) | |
| Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| at com.datastax.driver.core.ResultSetFuture.convertException (ResultSetFuture.java:243) | |
| com.datastax.driver.core.ResultSetFuture$ResponseCallback.onSet (ResultSetFuture.java:119) | |
| com.datastax.driver.core.RequestHandler.setFinalResult (RequestHandler.java:202) | |
| com.datastax.driver.core.RequestHandler.onSet (RequestHandler.java:331) | |
| com.datastax.driver.core.Connection$Dispatcher.messageReceived (Connection.java:484) | |
| org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream (SimpleChannelUpstreamHandler.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:296) | |
| org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream (OneToOneDecoder.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream (OneToOneDecoder.java:68) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:296) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived (FrameDecoder.java:462) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode (FrameDecoder.java:443) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived (FrameDecoder.java:303) | |
| org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream (SimpleChannelUpstreamHandler.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:555) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:268) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:255) | |
| org.jboss.netty.channel.socket.nio.NioWorker.read (NioWorker.java:88) | |
| org.jboss.netty.channel.socket.nio.AbstractNioWorker.process (AbstractNioWorker.java:107) | |
| org.jboss.netty.channel.socket.nio.AbstractNioSelector.run (AbstractNioSelector.java:312) | |
| org.jboss.netty.channel.socket.nio.AbstractNioWorker.run (AbstractNioWorker.java:88) | |
| org.jboss.netty.channel.socket.nio.NioWorker.run (NioWorker.java:178) | |
| org.jboss.netty.util.ThreadRenamingRunnable.run (ThreadRenamingRunnable.java:108) | |
| org.jboss.netty.util.internal.DeadLockProofWorker$1.run (DeadLockProofWorker.java:42) | |
| java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1145) | |
| java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:615) | |
| java.lang.Thread.run (Thread.java:722) | |
| 122 :ok 24 ▏ | |
| 123 :ok 24 ▏ | |
| 124 :ok 23 ▏ | |
| 125 :ok 21 ▏ | |
| 126 :ok 23 ▏ | |
| 127 :ok 26 ▏ | |
| 128 :ok 26 ▏ | |
| 129 :ok 24 ▏ | |
| 130 :ok 19 ▏ | |
| 131 :ok 17 ▏ | |
| 132 :ok 18 ▏ | |
| 133 :ok 19 ▏ | |
| 134 :ok 15 ▏ | |
| 135 :ok 19 ▏ | |
| 136 :ok 22 ▏ | |
| 137 :ok 23 ▏ | |
| 138 :ok 23 ▏ | |
| 139 :ok 23 ▏ | |
| 140 :ok 20 ▏ | |
| 141 :ok 25 ▏ | |
| 142 :error 10017 ████████████████████▏ | |
| Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| at com.datastax.driver.core.exceptions.WriteTimeoutException.copy (WriteTimeoutException.java:54) | |
| com.datastax.driver.core.ResultSetFuture.extractCauseFromExecutionException (ResultSetFuture.java:214) | |
| com.datastax.driver.core.ResultSetFuture.getUninterruptibly (ResultSetFuture.java:169) | |
| clojurewerkz.cassaforte.client$execute.doInvoke (client.clj:289) | |
| clojure.lang.RestFn.invoke (RestFn.java:457) | |
| jepsen.cassandra$isolation_app$reify__2559$fn__2560.invoke (cassandra.clj:186) | |
| jepsen.cassandra$isolation_app$reify__2559.add (cassandra.clj:177) | |
| jepsen.set_app$eval2420$fn__2443$G__2410__2446.invoke (set_app.clj:11) | |
| jepsen.set_app$eval2420$fn__2443$G__2409__2450.invoke (set_app.clj:11) | |
| clojure.lang.AFn.applyToHelper (AFn.java:163) | |
| clojure.lang.AFn.applyTo (AFn.java:151) | |
| clojure.core$apply.invoke (core.clj:619) | |
| clojure.core$partial$fn__4190.doInvoke (core.clj:2396) | |
| clojure.lang.RestFn.invoke (RestFn.java:408) | |
| jepsen.load$wrap_catch$catcher__2389.invoke (load.clj:110) | |
| jepsen.load$wrap_latency$measure_latency__2386.invoke (load.clj:100) | |
| jepsen.load$wrap_record_req$record_req__2398.invoke (load.clj:129) | |
| jepsen.console$wrap_ordered_log$logger__2346.invoke (console.clj:108) | |
| jepsen.load$map_fixed_rate$boss__2379$fn__2381.invoke (load.clj:65) | |
| clojure.lang.AFn.run (AFn.java:24) | |
| java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1145) | |
| java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:615) | |
| java.lang.Thread.run (Thread.java:722) | |
| Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| at com.datastax.driver.core.ResultSetFuture.convertException (ResultSetFuture.java:243) | |
| com.datastax.driver.core.ResultSetFuture$ResponseCallback.onSet (ResultSetFuture.java:119) | |
| com.datastax.driver.core.RequestHandler.setFinalResult (RequestHandler.java:202) | |
| com.datastax.driver.core.RequestHandler.onSet (RequestHandler.java:331) | |
| com.datastax.driver.core.Connection$Dispatcher.messageReceived (Connection.java:484) | |
| org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream (SimpleChannelUpstreamHandler.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:296) | |
| org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream (OneToOneDecoder.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream (OneToOneDecoder.java:68) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:296) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived (FrameDecoder.java:462) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode (FrameDecoder.java:443) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived (FrameDecoder.java:303) | |
| org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream (SimpleChannelUpstreamHandler.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:555) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:268) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:255) | |
| org.jboss.netty.channel.socket.nio.NioWorker.read (NioWorker.java:88) | |
| org.jboss.netty.channel.socket.nio.AbstractNioWorker.process (AbstractNioWorker.java:107) | |
| org.jboss.netty.channel.socket.nio.AbstractNioSelector.run (AbstractNioSelector.java:312) | |
| org.jboss.netty.channel.socket.nio.AbstractNioWorker.run (AbstractNioWorker.java:88) | |
| org.jboss.netty.channel.socket.nio.NioWorker.run (NioWorker.java:178) | |
| org.jboss.netty.util.ThreadRenamingRunnable.run (ThreadRenamingRunnable.java:108) | |
| org.jboss.netty.util.internal.DeadLockProofWorker$1.run (DeadLockProofWorker.java:42) | |
| java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1145) | |
| java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:615) | |
| java.lang.Thread.run (Thread.java:722) | |
| 143 :ok 28 ▏ | |
| 144 :ok 27 ▏ | |
| 145 :error 10025 ████████████████████▏ | |
| Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| at com.datastax.driver.core.exceptions.WriteTimeoutException.copy (WriteTimeoutException.java:54) | |
| com.datastax.driver.core.ResultSetFuture.extractCauseFromExecutionException (ResultSetFuture.java:214) | |
| com.datastax.driver.core.ResultSetFuture.getUninterruptibly (ResultSetFuture.java:169) | |
| clojurewerkz.cassaforte.client$execute.doInvoke (client.clj:289) | |
| clojure.lang.RestFn.invoke (RestFn.java:457) | |
| jepsen.cassandra$isolation_app$reify__2559$fn__2560.invoke (cassandra.clj:186) | |
| jepsen.cassandra$isolation_app$reify__2559.add (cassandra.clj:177) | |
| jepsen.set_app$eval2420$fn__2443$G__2410__2446.invoke (set_app.clj:11) | |
| jepsen.set_app$eval2420$fn__2443$G__2409__2450.invoke (set_app.clj:11) | |
| clojure.lang.AFn.applyToHelper (AFn.java:163) | |
| clojure.lang.AFn.applyTo (AFn.java:151) | |
| clojure.core$apply.invoke (core.clj:619) | |
| clojure.core$partial$fn__4190.doInvoke (core.clj:2396) | |
| clojure.lang.RestFn.invoke (RestFn.java:408) | |
| jepsen.load$wrap_catch$catcher__2389.invoke (load.clj:110) | |
| jepsen.load$wrap_latency$measure_latency__2386.invoke (load.clj:100) | |
| jepsen.load$wrap_record_req$record_req__2398.invoke (load.clj:129) | |
| jepsen.console$wrap_ordered_log$logger__2346.invoke (console.clj:108) | |
| jepsen.load$map_fixed_rate$boss__2379$fn__2381.invoke (load.clj:65) | |
| clojure.lang.AFn.run (AFn.java:24) | |
| java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1145) | |
| java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:615) | |
| java.lang.Thread.run (Thread.java:722) | |
| Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| at com.datastax.driver.core.ResultSetFuture.convertException (ResultSetFuture.java:243) | |
| com.datastax.driver.core.ResultSetFuture$ResponseCallback.onSet (ResultSetFuture.java:119) | |
| com.datastax.driver.core.RequestHandler.setFinalResult (RequestHandler.java:202) | |
| com.datastax.driver.core.RequestHandler.onSet (RequestHandler.java:331) | |
| com.datastax.driver.core.Connection$Dispatcher.messageReceived (Connection.java:484) | |
| org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream (SimpleChannelUpstreamHandler.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:296) | |
| org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream (OneToOneDecoder.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream (OneToOneDecoder.java:68) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:296) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived (FrameDecoder.java:462) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode (FrameDecoder.java:443) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived (FrameDecoder.java:303) | |
| org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream (SimpleChannelUpstreamHandler.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:555) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:268) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:255) | |
| org.jboss.netty.channel.socket.nio.NioWorker.read (NioWorker.java:88) | |
| org.jboss.netty.channel.socket.nio.AbstractNioWorker.process (AbstractNioWorker.java:107) | |
| org.jboss.netty.channel.socket.nio.AbstractNioSelector.run (AbstractNioSelector.java:312) | |
| org.jboss.netty.channel.socket.nio.AbstractNioWorker.run (AbstractNioWorker.java:88) | |
| org.jboss.netty.channel.socket.nio.NioWorker.run (NioWorker.java:178) | |
| org.jboss.netty.util.ThreadRenamingRunnable.run (ThreadRenamingRunnable.java:108) | |
| org.jboss.netty.util.internal.DeadLockProofWorker$1.run (DeadLockProofWorker.java:42) | |
| java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1145) | |
| java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:615) | |
| java.lang.Thread.run (Thread.java:722) | |
| 146 :error 10026 ████████████████████▏ | |
| Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| at com.datastax.driver.core.exceptions.WriteTimeoutException.copy (WriteTimeoutException.java:54) | |
| com.datastax.driver.core.ResultSetFuture.extractCauseFromExecutionException (ResultSetFuture.java:214) | |
| com.datastax.driver.core.ResultSetFuture.getUninterruptibly (ResultSetFuture.java:169) | |
| clojurewerkz.cassaforte.client$execute.doInvoke (client.clj:289) | |
| clojure.lang.RestFn.invoke (RestFn.java:457) | |
| jepsen.cassandra$isolation_app$reify__2559$fn__2560.invoke (cassandra.clj:186) | |
| jepsen.cassandra$isolation_app$reify__2559.add (cassandra.clj:177) | |
| jepsen.set_app$eval2420$fn__2443$G__2410__2446.invoke (set_app.clj:11) | |
| jepsen.set_app$eval2420$fn__2443$G__2409__2450.invoke (set_app.clj:11) | |
| clojure.lang.AFn.applyToHelper (AFn.java:163) | |
| clojure.lang.AFn.applyTo (AFn.java:151) | |
| clojure.core$apply.invoke (core.clj:619) | |
| clojure.core$partial$fn__4190.doInvoke (core.clj:2396) | |
| clojure.lang.RestFn.invoke (RestFn.java:408) | |
| jepsen.load$wrap_catch$catcher__2389.invoke (load.clj:110) | |
| jepsen.load$wrap_latency$measure_latency__2386.invoke (load.clj:100) | |
| jepsen.load$wrap_record_req$record_req__2398.invoke (load.clj:129) | |
| jepsen.console$wrap_ordered_log$logger__2346.invoke (console.clj:108) | |
| jepsen.load$map_fixed_rate$boss__2379$fn__2381.invoke (load.clj:65) | |
| clojure.lang.AFn.run (AFn.java:24) | |
| java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1145) | |
| java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:615) | |
| java.lang.Thread.run (Thread.java:722) | |
| Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| at com.datastax.driver.core.ResultSetFuture.convertException (ResultSetFuture.java:243) | |
| com.datastax.driver.core.ResultSetFuture$ResponseCallback.onSet (ResultSetFuture.java:119) | |
| com.datastax.driver.core.RequestHandler.setFinalResult (RequestHandler.java:202) | |
| com.datastax.driver.core.RequestHandler.onSet (RequestHandler.java:331) | |
| com.datastax.driver.core.Connection$Dispatcher.messageReceived (Connection.java:484) | |
| org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream (SimpleChannelUpstreamHandler.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:296) | |
| org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream (OneToOneDecoder.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream (OneToOneDecoder.java:68) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:296) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived (FrameDecoder.java:462) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode (FrameDecoder.java:443) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived (FrameDecoder.java:303) | |
| org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream (SimpleChannelUpstreamHandler.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:555) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:268) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:255) | |
| org.jboss.netty.channel.socket.nio.NioWorker.read (NioWorker.java:88) | |
| org.jboss.netty.channel.socket.nio.AbstractNioWorker.process (AbstractNioWorker.java:107) | |
| org.jboss.netty.channel.socket.nio.AbstractNioSelector.run (AbstractNioSelector.java:312) | |
| org.jboss.netty.channel.socket.nio.AbstractNioWorker.run (AbstractNioWorker.java:88) | |
| org.jboss.netty.channel.socket.nio.NioWorker.run (NioWorker.java:178) | |
| org.jboss.netty.util.ThreadRenamingRunnable.run (ThreadRenamingRunnable.java:108) | |
| org.jboss.netty.util.internal.DeadLockProofWorker$1.run (DeadLockProofWorker.java:42) | |
| java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1145) | |
| java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:615) | |
| java.lang.Thread.run (Thread.java:722) | |
| 147 :error 10015 ████████████████████▏ | |
| Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| at com.datastax.driver.core.exceptions.WriteTimeoutException.copy (WriteTimeoutException.java:54) | |
| com.datastax.driver.core.ResultSetFuture.extractCauseFromExecutionException (ResultSetFuture.java:214) | |
| com.datastax.driver.core.ResultSetFuture.getUninterruptibly (ResultSetFuture.java:169) | |
| clojurewerkz.cassaforte.client$execute.doInvoke (client.clj:289) | |
| clojure.lang.RestFn.invoke (RestFn.java:457) | |
| jepsen.cassandra$isolation_app$reify__2559$fn__2560.invoke (cassandra.clj:186) | |
| jepsen.cassandra$isolation_app$reify__2559.add (cassandra.clj:177) | |
| jepsen.set_app$eval2420$fn__2443$G__2410__2446.invoke (set_app.clj:11) | |
| jepsen.set_app$eval2420$fn__2443$G__2409__2450.invoke (set_app.clj:11) | |
| clojure.lang.AFn.applyToHelper (AFn.java:163) | |
| clojure.lang.AFn.applyTo (AFn.java:151) | |
| clojure.core$apply.invoke (core.clj:619) | |
| clojure.core$partial$fn__4190.doInvoke (core.clj:2396) | |
| clojure.lang.RestFn.invoke (RestFn.java:408) | |
| jepsen.load$wrap_catch$catcher__2389.invoke (load.clj:110) | |
| jepsen.load$wrap_latency$measure_latency__2386.invoke (load.clj:100) | |
| jepsen.load$wrap_record_req$record_req__2398.invoke (load.clj:129) | |
| jepsen.console$wrap_ordered_log$logger__2346.invoke (console.clj:108) | |
| jepsen.load$map_fixed_rate$boss__2379$fn__2381.invoke (load.clj:65) | |
| clojure.lang.AFn.run (AFn.java:24) | |
| java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1145) | |
| java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:615) | |
| java.lang.Thread.run (Thread.java:722) | |
| Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| at com.datastax.driver.core.ResultSetFuture.convertException (ResultSetFuture.java:243) | |
| com.datastax.driver.core.ResultSetFuture$ResponseCallback.onSet (ResultSetFuture.java:119) | |
| com.datastax.driver.core.RequestHandler.setFinalResult (RequestHandler.java:202) | |
| com.datastax.driver.core.RequestHandler.onSet (RequestHandler.java:331) | |
| com.datastax.driver.core.Connection$Dispatcher.messageReceived (Connection.java:484) | |
| org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream (SimpleChannelUpstreamHandler.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:296) | |
| org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream (OneToOneDecoder.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream (OneToOneDecoder.java:68) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:296) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived (FrameDecoder.java:462) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode (FrameDecoder.java:443) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived (FrameDecoder.java:303) | |
| org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream (SimpleChannelUpstreamHandler.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:555) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:268) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:255) | |
| org.jboss.netty.channel.socket.nio.NioWorker.read (NioWorker.java:88) | |
| org.jboss.netty.channel.socket.nio.AbstractNioWorker.process (AbstractNioWorker.java:107) | |
| org.jboss.netty.channel.socket.nio.AbstractNioSelector.run (AbstractNioSelector.java:312) | |
| org.jboss.netty.channel.socket.nio.AbstractNioWorker.run (AbstractNioWorker.java:88) | |
| org.jboss.netty.channel.socket.nio.NioWorker.run (NioWorker.java:178) | |
| org.jboss.netty.util.ThreadRenamingRunnable.run (ThreadRenamingRunnable.java:108) | |
| org.jboss.netty.util.internal.DeadLockProofWorker$1.run (DeadLockProofWorker.java:42) | |
| java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1145) | |
| java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:615) | |
| java.lang.Thread.run (Thread.java:722) | |
| 148 :error 10027 ████████████████████▏ | |
| Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| at com.datastax.driver.core.exceptions.WriteTimeoutException.copy (WriteTimeoutException.java:54) | |
| com.datastax.driver.core.ResultSetFuture.extractCauseFromExecutionException (ResultSetFuture.java:214) | |
| com.datastax.driver.core.ResultSetFuture.getUninterruptibly (ResultSetFuture.java:169) | |
| clojurewerkz.cassaforte.client$execute.doInvoke (client.clj:289) | |
| clojure.lang.RestFn.invoke (RestFn.java:457) | |
| jepsen.cassandra$isolation_app$reify__2559$fn__2560.invoke (cassandra.clj:186) | |
| jepsen.cassandra$isolation_app$reify__2559.add (cassandra.clj:177) | |
| jepsen.set_app$eval2420$fn__2443$G__2410__2446.invoke (set_app.clj:11) | |
| jepsen.set_app$eval2420$fn__2443$G__2409__2450.invoke (set_app.clj:11) | |
| clojure.lang.AFn.applyToHelper (AFn.java:163) | |
| clojure.lang.AFn.applyTo (AFn.java:151) | |
| clojure.core$apply.invoke (core.clj:619) | |
| clojure.core$partial$fn__4190.doInvoke (core.clj:2396) | |
| clojure.lang.RestFn.invoke (RestFn.java:408) | |
| jepsen.load$wrap_catch$catcher__2389.invoke (load.clj:110) | |
| jepsen.load$wrap_latency$measure_latency__2386.invoke (load.clj:100) | |
| jepsen.load$wrap_record_req$record_req__2398.invoke (load.clj:129) | |
| jepsen.console$wrap_ordered_log$logger__2346.invoke (console.clj:108) | |
| jepsen.load$map_fixed_rate$boss__2379$fn__2381.invoke (load.clj:65) | |
| clojure.lang.AFn.run (AFn.java:24) | |
| java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1145) | |
| java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:615) | |
| java.lang.Thread.run (Thread.java:722) | |
| Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| at com.datastax.driver.core.ResultSetFuture.convertException (ResultSetFuture.java:243) | |
| com.datastax.driver.core.ResultSetFuture$ResponseCallback.onSet (ResultSetFuture.java:119) | |
| com.datastax.driver.core.RequestHandler.setFinalResult (RequestHandler.java:202) | |
| com.datastax.driver.core.RequestHandler.onSet (RequestHandler.java:331) | |
| com.datastax.driver.core.Connection$Dispatcher.messageReceived (Connection.java:484) | |
| org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream (SimpleChannelUpstreamHandler.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:296) | |
| org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream (OneToOneDecoder.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream (OneToOneDecoder.java:68) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:296) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived (FrameDecoder.java:462) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode (FrameDecoder.java:443) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived (FrameDecoder.java:303) | |
| org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream (SimpleChannelUpstreamHandler.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:555) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:268) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:255) | |
| org.jboss.netty.channel.socket.nio.NioWorker.read (NioWorker.java:88) | |
| org.jboss.netty.channel.socket.nio.AbstractNioWorker.process (AbstractNioWorker.java:107) | |
| org.jboss.netty.channel.socket.nio.AbstractNioSelector.run (AbstractNioSelector.java:312) | |
| org.jboss.netty.channel.socket.nio.AbstractNioWorker.run (AbstractNioWorker.java:88) | |
| org.jboss.netty.channel.socket.nio.NioWorker.run (NioWorker.java:178) | |
| org.jboss.netty.util.ThreadRenamingRunnable.run (ThreadRenamingRunnable.java:108) | |
| org.jboss.netty.util.internal.DeadLockProofWorker$1.run (DeadLockProofWorker.java:42) | |
| java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1145) | |
| java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:615) | |
| java.lang.Thread.run (Thread.java:722) | |
| 149 :ok 18 ▏ | |
| 150 :ok 22 ▏ | |
| 151 :error 10028 ████████████████████▏ | |
| Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| at com.datastax.driver.core.exceptions.WriteTimeoutException.copy (WriteTimeoutException.java:54) | |
| com.datastax.driver.core.ResultSetFuture.extractCauseFromExecutionException (ResultSetFuture.java:214) | |
| com.datastax.driver.core.ResultSetFuture.getUninterruptibly (ResultSetFuture.java:169) | |
| clojurewerkz.cassaforte.client$execute.doInvoke (client.clj:289) | |
| clojure.lang.RestFn.invoke (RestFn.java:457) | |
| jepsen.cassandra$isolation_app$reify__2559$fn__2560.invoke (cassandra.clj:186) | |
| jepsen.cassandra$isolation_app$reify__2559.add (cassandra.clj:177) | |
| jepsen.set_app$eval2420$fn__2443$G__2410__2446.invoke (set_app.clj:11) | |
| jepsen.set_app$eval2420$fn__2443$G__2409__2450.invoke (set_app.clj:11) | |
| clojure.lang.AFn.applyToHelper (AFn.java:163) | |
| clojure.lang.AFn.applyTo (AFn.java:151) | |
| clojure.core$apply.invoke (core.clj:619) | |
| clojure.core$partial$fn__4190.doInvoke (core.clj:2396) | |
| clojure.lang.RestFn.invoke (RestFn.java:408) | |
| jepsen.load$wrap_catch$catcher__2389.invoke (load.clj:110) | |
| jepsen.load$wrap_latency$measure_latency__2386.invoke (load.clj:100) | |
| jepsen.load$wrap_record_req$record_req__2398.invoke (load.clj:129) | |
| jepsen.console$wrap_ordered_log$logger__2346.invoke (console.clj:108) | |
| jepsen.load$map_fixed_rate$boss__2379$fn__2381.invoke (load.clj:65) | |
| clojure.lang.AFn.run (AFn.java:24) | |
| java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1145) | |
| java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:615) | |
| java.lang.Thread.run (Thread.java:722) | |
| Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| at com.datastax.driver.core.ResultSetFuture.convertException (ResultSetFuture.java:243) | |
| com.datastax.driver.core.ResultSetFuture$ResponseCallback.onSet (ResultSetFuture.java:119) | |
| com.datastax.driver.core.RequestHandler.setFinalResult (RequestHandler.java:202) | |
| com.datastax.driver.core.RequestHandler.onSet (RequestHandler.java:331) | |
| com.datastax.driver.core.Connection$Dispatcher.messageReceived (Connection.java:484) | |
| org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream (SimpleChannelUpstreamHandler.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:296) | |
| org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream (OneToOneDecoder.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream (OneToOneDecoder.java:68) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:296) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived (FrameDecoder.java:462) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode (FrameDecoder.java:443) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived (FrameDecoder.java:303) | |
| org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream (SimpleChannelUpstreamHandler.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:555) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:268) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:255) | |
| org.jboss.netty.channel.socket.nio.NioWorker.read (NioWorker.java:88) | |
| org.jboss.netty.channel.socket.nio.AbstractNioWorker.process (AbstractNioWorker.java:107) | |
| org.jboss.netty.channel.socket.nio.AbstractNioSelector.run (AbstractNioSelector.java:312) | |
| org.jboss.netty.channel.socket.nio.AbstractNioWorker.run (AbstractNioWorker.java:88) | |
| org.jboss.netty.channel.socket.nio.NioWorker.run (NioWorker.java:178) | |
| org.jboss.netty.util.ThreadRenamingRunnable.run (ThreadRenamingRunnable.java:108) | |
| org.jboss.netty.util.internal.DeadLockProofWorker$1.run (DeadLockProofWorker.java:42) | |
| java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1145) | |
| java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:615) | |
| java.lang.Thread.run (Thread.java:722) | |
| 152 :ok 22 ▏ | |
| 153 :ok 21 ▏ | |
| 154 :ok 19 ▏ | |
| 155 :ok 23 ▏ | |
| 156 :error 10025 ████████████████████▏ | |
| Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| at com.datastax.driver.core.exceptions.WriteTimeoutException.copy (WriteTimeoutException.java:54) | |
| com.datastax.driver.core.ResultSetFuture.extractCauseFromExecutionException (ResultSetFuture.java:214) | |
| com.datastax.driver.core.ResultSetFuture.getUninterruptibly (ResultSetFuture.java:169) | |
| clojurewerkz.cassaforte.client$execute.doInvoke (client.clj:289) | |
| clojure.lang.RestFn.invoke (RestFn.java:457) | |
| jepsen.cassandra$isolation_app$reify__2559$fn__2560.invoke (cassandra.clj:186) | |
| jepsen.cassandra$isolation_app$reify__2559.add (cassandra.clj:177) | |
| jepsen.set_app$eval2420$fn__2443$G__2410__2446.invoke (set_app.clj:11) | |
| jepsen.set_app$eval2420$fn__2443$G__2409__2450.invoke (set_app.clj:11) | |
| clojure.lang.AFn.applyToHelper (AFn.java:163) | |
| clojure.lang.AFn.applyTo (AFn.java:151) | |
| clojure.core$apply.invoke (core.clj:619) | |
| clojure.core$partial$fn__4190.doInvoke (core.clj:2396) | |
| clojure.lang.RestFn.invoke (RestFn.java:408) | |
| jepsen.load$wrap_catch$catcher__2389.invoke (load.clj:110) | |
| jepsen.load$wrap_latency$measure_latency__2386.invoke (load.clj:100) | |
| jepsen.load$wrap_record_req$record_req__2398.invoke (load.clj:129) | |
| jepsen.console$wrap_ordered_log$logger__2346.invoke (console.clj:108) | |
| jepsen.load$map_fixed_rate$boss__2379$fn__2381.invoke (load.clj:65) | |
| clojure.lang.AFn.run (AFn.java:24) | |
| java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1145) | |
| java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:615) | |
| java.lang.Thread.run (Thread.java:722) | |
| Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| at com.datastax.driver.core.ResultSetFuture.convertException (ResultSetFuture.java:243) | |
| com.datastax.driver.core.ResultSetFuture$ResponseCallback.onSet (ResultSetFuture.java:119) | |
| com.datastax.driver.core.RequestHandler.setFinalResult (RequestHandler.java:202) | |
| com.datastax.driver.core.RequestHandler.onSet (RequestHandler.java:331) | |
| com.datastax.driver.core.Connection$Dispatcher.messageReceived (Connection.java:484) | |
| org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream (SimpleChannelUpstreamHandler.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:296) | |
| org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream (OneToOneDecoder.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream (OneToOneDecoder.java:68) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:296) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived (FrameDecoder.java:462) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode (FrameDecoder.java:443) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived (FrameDecoder.java:303) | |
| org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream (SimpleChannelUpstreamHandler.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:555) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:268) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:255) | |
| org.jboss.netty.channel.socket.nio.NioWorker.read (NioWorker.java:88) | |
| org.jboss.netty.channel.socket.nio.AbstractNioWorker.process (AbstractNioWorker.java:107) | |
| org.jboss.netty.channel.socket.nio.AbstractNioSelector.run (AbstractNioSelector.java:312) | |
| org.jboss.netty.channel.socket.nio.AbstractNioWorker.run (AbstractNioWorker.java:88) | |
| org.jboss.netty.channel.socket.nio.NioWorker.run (NioWorker.java:178) | |
| org.jboss.netty.util.ThreadRenamingRunnable.run (ThreadRenamingRunnable.java:108) | |
| org.jboss.netty.util.internal.DeadLockProofWorker$1.run (DeadLockProofWorker.java:42) | |
| java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1145) | |
| java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:615) | |
| java.lang.Thread.run (Thread.java:722) | |
| 157 :error 10029 ████████████████████▏ | |
| Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| at com.datastax.driver.core.exceptions.WriteTimeoutException.copy (WriteTimeoutException.java:54) | |
| com.datastax.driver.core.ResultSetFuture.extractCauseFromExecutionException (ResultSetFuture.java:214) | |
| com.datastax.driver.core.ResultSetFuture.getUninterruptibly (ResultSetFuture.java:169) | |
| clojurewerkz.cassaforte.client$execute.doInvoke (client.clj:289) | |
| clojure.lang.RestFn.invoke (RestFn.java:457) | |
| jepsen.cassandra$isolation_app$reify__2559$fn__2560.invoke (cassandra.clj:186) | |
| jepsen.cassandra$isolation_app$reify__2559.add (cassandra.clj:177) | |
| jepsen.set_app$eval2420$fn__2443$G__2410__2446.invoke (set_app.clj:11) | |
| jepsen.set_app$eval2420$fn__2443$G__2409__2450.invoke (set_app.clj:11) | |
| clojure.lang.AFn.applyToHelper (AFn.java:163) | |
| clojure.lang.AFn.applyTo (AFn.java:151) | |
| clojure.core$apply.invoke (core.clj:619) | |
| clojure.core$partial$fn__4190.doInvoke (core.clj:2396) | |
| clojure.lang.RestFn.invoke (RestFn.java:408) | |
| jepsen.load$wrap_catch$catcher__2389.invoke (load.clj:110) | |
| jepsen.load$wrap_latency$measure_latency__2386.invoke (load.clj:100) | |
| jepsen.load$wrap_record_req$record_req__2398.invoke (load.clj:129) | |
| jepsen.console$wrap_ordered_log$logger__2346.invoke (console.clj:108) | |
| jepsen.load$map_fixed_rate$boss__2379$fn__2381.invoke (load.clj:65) | |
| clojure.lang.AFn.run (AFn.java:24) | |
| java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1145) | |
| java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:615) | |
| java.lang.Thread.run (Thread.java:722) | |
| Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| at com.datastax.driver.core.ResultSetFuture.convertException (ResultSetFuture.java:243) | |
| com.datastax.driver.core.ResultSetFuture$ResponseCallback.onSet (ResultSetFuture.java:119) | |
| com.datastax.driver.core.RequestHandler.setFinalResult (RequestHandler.java:202) | |
| com.datastax.driver.core.RequestHandler.onSet (RequestHandler.java:331) | |
| com.datastax.driver.core.Connection$Dispatcher.messageReceived (Connection.java:484) | |
| org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream (SimpleChannelUpstreamHandler.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:296) | |
| org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream (OneToOneDecoder.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream (OneToOneDecoder.java:68) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:296) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived (FrameDecoder.java:462) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode (FrameDecoder.java:443) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived (FrameDecoder.java:303) | |
| org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream (SimpleChannelUpstreamHandler.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:555) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:268) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:255) | |
| org.jboss.netty.channel.socket.nio.NioWorker.read (NioWorker.java:88) | |
| org.jboss.netty.channel.socket.nio.AbstractNioWorker.process (AbstractNioWorker.java:107) | |
| org.jboss.netty.channel.socket.nio.AbstractNioSelector.run (AbstractNioSelector.java:312) | |
| org.jboss.netty.channel.socket.nio.AbstractNioWorker.run (AbstractNioWorker.java:88) | |
| org.jboss.netty.channel.socket.nio.NioWorker.run (NioWorker.java:178) | |
| org.jboss.netty.util.ThreadRenamingRunnable.run (ThreadRenamingRunnable.java:108) | |
| org.jboss.netty.util.internal.DeadLockProofWorker$1.run (DeadLockProofWorker.java:42) | |
| java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1145) | |
| java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:615) | |
| java.lang.Thread.run (Thread.java:722) | |
| 158 :ok 27 ▏ | |
| 159 :ok 24 ▏ | |
| 160 :ok 16 ▏ | |
| 161 :ok 16 ▏ | |
| 162 :error 10022 ████████████████████▏ | |
| Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| at com.datastax.driver.core.exceptions.WriteTimeoutException.copy (WriteTimeoutException.java:54) | |
| com.datastax.driver.core.ResultSetFuture.extractCauseFromExecutionException (ResultSetFuture.java:214) | |
| com.datastax.driver.core.ResultSetFuture.getUninterruptibly (ResultSetFuture.java:169) | |
| clojurewerkz.cassaforte.client$execute.doInvoke (client.clj:289) | |
| clojure.lang.RestFn.invoke (RestFn.java:457) | |
| jepsen.cassandra$isolation_app$reify__2559$fn__2560.invoke (cassandra.clj:186) | |
| jepsen.cassandra$isolation_app$reify__2559.add (cassandra.clj:177) | |
| jepsen.set_app$eval2420$fn__2443$G__2410__2446.invoke (set_app.clj:11) | |
| jepsen.set_app$eval2420$fn__2443$G__2409__2450.invoke (set_app.clj:11) | |
| clojure.lang.AFn.applyToHelper (AFn.java:163) | |
| clojure.lang.AFn.applyTo (AFn.java:151) | |
| clojure.core$apply.invoke (core.clj:619) | |
| clojure.core$partial$fn__4190.doInvoke (core.clj:2396) | |
| clojure.lang.RestFn.invoke (RestFn.java:408) | |
| jepsen.load$wrap_catch$catcher__2389.invoke (load.clj:110) | |
| jepsen.load$wrap_latency$measure_latency__2386.invoke (load.clj:100) | |
| jepsen.load$wrap_record_req$record_req__2398.invoke (load.clj:129) | |
| jepsen.console$wrap_ordered_log$logger__2346.invoke (console.clj:108) | |
| jepsen.load$map_fixed_rate$boss__2379$fn__2381.invoke (load.clj:65) | |
| clojure.lang.AFn.run (AFn.java:24) | |
| java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1145) | |
| java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:615) | |
| java.lang.Thread.run (Thread.java:722) | |
| Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| at com.datastax.driver.core.ResultSetFuture.convertException (ResultSetFuture.java:243) | |
| com.datastax.driver.core.ResultSetFuture$ResponseCallback.onSet (ResultSetFuture.java:119) | |
| com.datastax.driver.core.RequestHandler.setFinalResult (RequestHandler.java:202) | |
| com.datastax.driver.core.RequestHandler.onSet (RequestHandler.java:331) | |
| com.datastax.driver.core.Connection$Dispatcher.messageReceived (Connection.java:484) | |
| org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream (SimpleChannelUpstreamHandler.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:296) | |
| org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream (OneToOneDecoder.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream (OneToOneDecoder.java:68) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:296) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived (FrameDecoder.java:462) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode (FrameDecoder.java:443) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived (FrameDecoder.java:303) | |
| org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream (SimpleChannelUpstreamHandler.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:555) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:268) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:255) | |
| org.jboss.netty.channel.socket.nio.NioWorker.read (NioWorker.java:88) | |
| org.jboss.netty.channel.socket.nio.AbstractNioWorker.process (AbstractNioWorker.java:107) | |
| org.jboss.netty.channel.socket.nio.AbstractNioSelector.run (AbstractNioSelector.java:312) | |
| org.jboss.netty.channel.socket.nio.AbstractNioWorker.run (AbstractNioWorker.java:88) | |
| org.jboss.netty.channel.socket.nio.NioWorker.run (NioWorker.java:178) | |
| org.jboss.netty.util.ThreadRenamingRunnable.run (ThreadRenamingRunnable.java:108) | |
| org.jboss.netty.util.internal.DeadLockProofWorker$1.run (DeadLockProofWorker.java:42) | |
| java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1145) | |
| java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:615) | |
| java.lang.Thread.run (Thread.java:722) | |
| 163 :ok 19 ▏ | |
| 164 :ok 21 ▏ | |
| 165 :ok 20 ▏ | |
| 166 :ok 19 ▏ | |
| 167 :error 10021 ████████████████████▏ | |
| Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| at com.datastax.driver.core.exceptions.WriteTimeoutException.copy (WriteTimeoutException.java:54) | |
| com.datastax.driver.core.ResultSetFuture.extractCauseFromExecutionException (ResultSetFuture.java:214) | |
| com.datastax.driver.core.ResultSetFuture.getUninterruptibly (ResultSetFuture.java:169) | |
| clojurewerkz.cassaforte.client$execute.doInvoke (client.clj:289) | |
| clojure.lang.RestFn.invoke (RestFn.java:457) | |
| jepsen.cassandra$isolation_app$reify__2559$fn__2560.invoke (cassandra.clj:186) | |
| jepsen.cassandra$isolation_app$reify__2559.add (cassandra.clj:177) | |
| jepsen.set_app$eval2420$fn__2443$G__2410__2446.invoke (set_app.clj:11) | |
| jepsen.set_app$eval2420$fn__2443$G__2409__2450.invoke (set_app.clj:11) | |
| clojure.lang.AFn.applyToHelper (AFn.java:163) | |
| clojure.lang.AFn.applyTo (AFn.java:151) | |
| clojure.core$apply.invoke (core.clj:619) | |
| clojure.core$partial$fn__4190.doInvoke (core.clj:2396) | |
| clojure.lang.RestFn.invoke (RestFn.java:408) | |
| jepsen.load$wrap_catch$catcher__2389.invoke (load.clj:110) | |
| jepsen.load$wrap_latency$measure_latency__2386.invoke (load.clj:100) | |
| jepsen.load$wrap_record_req$record_req__2398.invoke (load.clj:129) | |
| jepsen.console$wrap_ordered_log$logger__2346.invoke (console.clj:108) | |
| jepsen.load$map_fixed_rate$boss__2379$fn__2381.invoke (load.clj:65) | |
| clojure.lang.AFn.run (AFn.java:24) | |
| java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1145) | |
| java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:615) | |
| java.lang.Thread.run (Thread.java:722) | |
| Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| at com.datastax.driver.core.ResultSetFuture.convertException (ResultSetFuture.java:243) | |
| com.datastax.driver.core.ResultSetFuture$ResponseCallback.onSet (ResultSetFuture.java:119) | |
| com.datastax.driver.core.RequestHandler.setFinalResult (RequestHandler.java:202) | |
| com.datastax.driver.core.RequestHandler.onSet (RequestHandler.java:331) | |
| com.datastax.driver.core.Connection$Dispatcher.messageReceived (Connection.java:484) | |
| org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream (SimpleChannelUpstreamHandler.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:296) | |
| org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream (OneToOneDecoder.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream (OneToOneDecoder.java:68) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:296) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived (FrameDecoder.java:462) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode (FrameDecoder.java:443) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived (FrameDecoder.java:303) | |
| org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream (SimpleChannelUpstreamHandler.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:555) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:268) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:255) | |
| org.jboss.netty.channel.socket.nio.NioWorker.read (NioWorker.java:88) | |
| org.jboss.netty.channel.socket.nio.AbstractNioWorker.process (AbstractNioWorker.java:107) | |
| org.jboss.netty.channel.socket.nio.AbstractNioSelector.run (AbstractNioSelector.java:312) | |
| org.jboss.netty.channel.socket.nio.AbstractNioWorker.run (AbstractNioWorker.java:88) | |
| org.jboss.netty.channel.socket.nio.NioWorker.run (NioWorker.java:178) | |
| org.jboss.netty.util.ThreadRenamingRunnable.run (ThreadRenamingRunnable.java:108) | |
| org.jboss.netty.util.internal.DeadLockProofWorker$1.run (DeadLockProofWorker.java:42) | |
| java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1145) | |
| java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:615) | |
| java.lang.Thread.run (Thread.java:722) | |
| 168 :error 10027 ████████████████████▏ | |
| Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| at com.datastax.driver.core.exceptions.WriteTimeoutException.copy (WriteTimeoutException.java:54) | |
| com.datastax.driver.core.ResultSetFuture.extractCauseFromExecutionException (ResultSetFuture.java:214) | |
| com.datastax.driver.core.ResultSetFuture.getUninterruptibly (ResultSetFuture.java:169) | |
| clojurewerkz.cassaforte.client$execute.doInvoke (client.clj:289) | |
| clojure.lang.RestFn.invoke (RestFn.java:457) | |
| jepsen.cassandra$isolation_app$reify__2559$fn__2560.invoke (cassandra.clj:186) | |
| jepsen.cassandra$isolation_app$reify__2559.add (cassandra.clj:177) | |
| jepsen.set_app$eval2420$fn__2443$G__2410__2446.invoke (set_app.clj:11) | |
| jepsen.set_app$eval2420$fn__2443$G__2409__2450.invoke (set_app.clj:11) | |
| clojure.lang.AFn.applyToHelper (AFn.java:163) | |
| clojure.lang.AFn.applyTo (AFn.java:151) | |
| clojure.core$apply.invoke (core.clj:619) | |
| clojure.core$partial$fn__4190.doInvoke (core.clj:2396) | |
| clojure.lang.RestFn.invoke (RestFn.java:408) | |
| jepsen.load$wrap_catch$catcher__2389.invoke (load.clj:110) | |
| jepsen.load$wrap_latency$measure_latency__2386.invoke (load.clj:100) | |
| jepsen.load$wrap_record_req$record_req__2398.invoke (load.clj:129) | |
| jepsen.console$wrap_ordered_log$logger__2346.invoke (console.clj:108) | |
| jepsen.load$map_fixed_rate$boss__2379$fn__2381.invoke (load.clj:65) | |
| clojure.lang.AFn.run (AFn.java:24) | |
| java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1145) | |
| java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:615) | |
| java.lang.Thread.run (Thread.java:722) | |
| Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| at com.datastax.driver.core.ResultSetFuture.convertException (ResultSetFuture.java:243) | |
| com.datastax.driver.core.ResultSetFuture$ResponseCallback.onSet (ResultSetFuture.java:119) | |
| com.datastax.driver.core.RequestHandler.setFinalResult (RequestHandler.java:202) | |
| com.datastax.driver.core.RequestHandler.onSet (RequestHandler.java:331) | |
| com.datastax.driver.core.Connection$Dispatcher.messageReceived (Connection.java:484) | |
| org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream (SimpleChannelUpstreamHandler.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:296) | |
| org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream (OneToOneDecoder.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream (OneToOneDecoder.java:68) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:296) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived (FrameDecoder.java:462) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode (FrameDecoder.java:443) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived (FrameDecoder.java:303) | |
| org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream (SimpleChannelUpstreamHandler.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:555) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:268) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:255) | |
| org.jboss.netty.channel.socket.nio.NioWorker.read (NioWorker.java:88) | |
| org.jboss.netty.channel.socket.nio.AbstractNioWorker.process (AbstractNioWorker.java:107) | |
| org.jboss.netty.channel.socket.nio.AbstractNioSelector.run (AbstractNioSelector.java:312) | |
| org.jboss.netty.channel.socket.nio.AbstractNioWorker.run (AbstractNioWorker.java:88) | |
| org.jboss.netty.channel.socket.nio.NioWorker.run (NioWorker.java:178) | |
| org.jboss.netty.util.ThreadRenamingRunnable.run (ThreadRenamingRunnable.java:108) | |
| org.jboss.netty.util.internal.DeadLockProofWorker$1.run (DeadLockProofWorker.java:42) | |
| java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1145) | |
| java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:615) | |
| java.lang.Thread.run (Thread.java:722) | |
| 169 :ok 18 ▏ | |
| 170 :ok 21 ▏ | |
| 171 :ok 26 ▏ | |
| 172 :ok 21 ▏ | |
| 173 :ok 24 ▏ | |
| 174 :ok 25 ▏ | |
| 175 :ok 19 ▏ | |
| 176 :ok 19 ▏ | |
| 177 :error 10024 ████████████████████▏ | |
| Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| at com.datastax.driver.core.exceptions.WriteTimeoutException.copy (WriteTimeoutException.java:54) | |
| com.datastax.driver.core.ResultSetFuture.extractCauseFromExecutionException (ResultSetFuture.java:214) | |
| com.datastax.driver.core.ResultSetFuture.getUninterruptibly (ResultSetFuture.java:169) | |
| clojurewerkz.cassaforte.client$execute.doInvoke (client.clj:289) | |
| clojure.lang.RestFn.invoke (RestFn.java:457) | |
| jepsen.cassandra$isolation_app$reify__2559$fn__2560.invoke (cassandra.clj:186) | |
| jepsen.cassandra$isolation_app$reify__2559.add (cassandra.clj:177) | |
| jepsen.set_app$eval2420$fn__2443$G__2410__2446.invoke (set_app.clj:11) | |
| jepsen.set_app$eval2420$fn__2443$G__2409__2450.invoke (set_app.clj:11) | |
| clojure.lang.AFn.applyToHelper (AFn.java:163) | |
| clojure.lang.AFn.applyTo (AFn.java:151) | |
| clojure.core$apply.invoke (core.clj:619) | |
| clojure.core$partial$fn__4190.doInvoke (core.clj:2396) | |
| clojure.lang.RestFn.invoke (RestFn.java:408) | |
| jepsen.load$wrap_catch$catcher__2389.invoke (load.clj:110) | |
| jepsen.load$wrap_latency$measure_latency__2386.invoke (load.clj:100) | |
| jepsen.load$wrap_record_req$record_req__2398.invoke (load.clj:129) | |
| jepsen.console$wrap_ordered_log$logger__2346.invoke (console.clj:108) | |
| jepsen.load$map_fixed_rate$boss__2379$fn__2381.invoke (load.clj:65) | |
| clojure.lang.AFn.run (AFn.java:24) | |
| java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1145) | |
| java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:615) | |
| java.lang.Thread.run (Thread.java:722) | |
| Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| at com.datastax.driver.core.ResultSetFuture.convertException (ResultSetFuture.java:243) | |
| com.datastax.driver.core.ResultSetFuture$ResponseCallback.onSet (ResultSetFuture.java:119) | |
| com.datastax.driver.core.RequestHandler.setFinalResult (RequestHandler.java:202) | |
| com.datastax.driver.core.RequestHandler.onSet (RequestHandler.java:331) | |
| com.datastax.driver.core.Connection$Dispatcher.messageReceived (Connection.java:484) | |
| org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream (SimpleChannelUpstreamHandler.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:296) | |
| org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream (OneToOneDecoder.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream (OneToOneDecoder.java:68) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:296) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived (FrameDecoder.java:462) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode (FrameDecoder.java:443) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived (FrameDecoder.java:303) | |
| org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream (SimpleChannelUpstreamHandler.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:555) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:268) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:255) | |
| org.jboss.netty.channel.socket.nio.NioWorker.read (NioWorker.java:88) | |
| org.jboss.netty.channel.socket.nio.AbstractNioWorker.process (AbstractNioWorker.java:107) | |
| org.jboss.netty.channel.socket.nio.AbstractNioSelector.run (AbstractNioSelector.java:312) | |
| org.jboss.netty.channel.socket.nio.AbstractNioWorker.run (AbstractNioWorker.java:88) | |
| org.jboss.netty.channel.socket.nio.NioWorker.run (NioWorker.java:178) | |
| org.jboss.netty.util.ThreadRenamingRunnable.run (ThreadRenamingRunnable.java:108) | |
| org.jboss.netty.util.internal.DeadLockProofWorker$1.run (DeadLockProofWorker.java:42) | |
| java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1145) | |
| java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:615) | |
| java.lang.Thread.run (Thread.java:722) | |
| 178 :ok 18 ▏ | |
| 179 :ok 19 ▏ | |
| 180 :ok 18 ▏ | |
| 181 :ok 20 ▏ | |
| 182 :error 10023 ████████████████████▏ | |
| Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| at com.datastax.driver.core.exceptions.WriteTimeoutException.copy (WriteTimeoutException.java:54) | |
| com.datastax.driver.core.ResultSetFuture.extractCauseFromExecutionException (ResultSetFuture.java:214) | |
| com.datastax.driver.core.ResultSetFuture.getUninterruptibly (ResultSetFuture.java:169) | |
| clojurewerkz.cassaforte.client$execute.doInvoke (client.clj:289) | |
| clojure.lang.RestFn.invoke (RestFn.java:457) | |
| jepsen.cassandra$isolation_app$reify__2559$fn__2560.invoke (cassandra.clj:186) | |
| jepsen.cassandra$isolation_app$reify__2559.add (cassandra.clj:177) | |
| jepsen.set_app$eval2420$fn__2443$G__2410__2446.invoke (set_app.clj:11) | |
| jepsen.set_app$eval2420$fn__2443$G__2409__2450.invoke (set_app.clj:11) | |
| clojure.lang.AFn.applyToHelper (AFn.java:163) | |
| clojure.lang.AFn.applyTo (AFn.java:151) | |
| clojure.core$apply.invoke (core.clj:619) | |
| clojure.core$partial$fn__4190.doInvoke (core.clj:2396) | |
| clojure.lang.RestFn.invoke (RestFn.java:408) | |
| jepsen.load$wrap_catch$catcher__2389.invoke (load.clj:110) | |
| jepsen.load$wrap_latency$measure_latency__2386.invoke (load.clj:100) | |
| jepsen.load$wrap_record_req$record_req__2398.invoke (load.clj:129) | |
| jepsen.console$wrap_ordered_log$logger__2346.invoke (console.clj:108) | |
| jepsen.load$map_fixed_rate$boss__2379$fn__2381.invoke (load.clj:65) | |
| clojure.lang.AFn.run (AFn.java:24) | |
| java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1145) | |
| java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:615) | |
| java.lang.Thread.run (Thread.java:722) | |
| Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| at com.datastax.driver.core.ResultSetFuture.convertException (ResultSetFuture.java:243) | |
| com.datastax.driver.core.ResultSetFuture$ResponseCallback.onSet (ResultSetFuture.java:119) | |
| com.datastax.driver.core.RequestHandler.setFinalResult (RequestHandler.java:202) | |
| com.datastax.driver.core.RequestHandler.onSet (RequestHandler.java:331) | |
| com.datastax.driver.core.Connection$Dispatcher.messageReceived (Connection.java:484) | |
| org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream (SimpleChannelUpstreamHandler.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:296) | |
| org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream (OneToOneDecoder.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream (OneToOneDecoder.java:68) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:296) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived (FrameDecoder.java:462) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode (FrameDecoder.java:443) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived (FrameDecoder.java:303) | |
| org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream (SimpleChannelUpstreamHandler.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:555) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:268) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:255) | |
| org.jboss.netty.channel.socket.nio.NioWorker.read (NioWorker.java:88) | |
| org.jboss.netty.channel.socket.nio.AbstractNioWorker.process (AbstractNioWorker.java:107) | |
| org.jboss.netty.channel.socket.nio.AbstractNioSelector.run (AbstractNioSelector.java:312) | |
| org.jboss.netty.channel.socket.nio.AbstractNioWorker.run (AbstractNioWorker.java:88) | |
| org.jboss.netty.channel.socket.nio.NioWorker.run (NioWorker.java:178) | |
| org.jboss.netty.util.ThreadRenamingRunnable.run (ThreadRenamingRunnable.java:108) | |
| org.jboss.netty.util.internal.DeadLockProofWorker$1.run (DeadLockProofWorker.java:42) | |
| java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1145) | |
| java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:615) | |
| java.lang.Thread.run (Thread.java:722) | |
| 183 :ok 20 ▏ | |
| 184 :ok 17 ▏ | |
| 185 :ok 20 ▏ | |
| 186 :error 10014 ████████████████████▏ | |
| Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| at com.datastax.driver.core.exceptions.WriteTimeoutException.copy (WriteTimeoutException.java:54) | |
| com.datastax.driver.core.ResultSetFuture.extractCauseFromExecutionException (ResultSetFuture.java:214) | |
| com.datastax.driver.core.ResultSetFuture.getUninterruptibly (ResultSetFuture.java:169) | |
| clojurewerkz.cassaforte.client$execute.doInvoke (client.clj:289) | |
| clojure.lang.RestFn.invoke (RestFn.java:457) | |
| jepsen.cassandra$isolation_app$reify__2559$fn__2560.invoke (cassandra.clj:186) | |
| jepsen.cassandra$isolation_app$reify__2559.add (cassandra.clj:177) | |
| jepsen.set_app$eval2420$fn__2443$G__2410__2446.invoke (set_app.clj:11) | |
| jepsen.set_app$eval2420$fn__2443$G__2409__2450.invoke (set_app.clj:11) | |
| clojure.lang.AFn.applyToHelper (AFn.java:163) | |
| clojure.lang.AFn.applyTo (AFn.java:151) | |
| clojure.core$apply.invoke (core.clj:619) | |
| clojure.core$partial$fn__4190.doInvoke (core.clj:2396) | |
| clojure.lang.RestFn.invoke (RestFn.java:408) | |
| jepsen.load$wrap_catch$catcher__2389.invoke (load.clj:110) | |
| jepsen.load$wrap_latency$measure_latency__2386.invoke (load.clj:100) | |
| jepsen.load$wrap_record_req$record_req__2398.invoke (load.clj:129) | |
| jepsen.console$wrap_ordered_log$logger__2346.invoke (console.clj:108) | |
| jepsen.load$map_fixed_rate$boss__2379$fn__2381.invoke (load.clj:65) | |
| clojure.lang.AFn.run (AFn.java:24) | |
| java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1145) | |
| java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:615) | |
| java.lang.Thread.run (Thread.java:722) | |
| Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| at com.datastax.driver.core.ResultSetFuture.convertException (ResultSetFuture.java:243) | |
| com.datastax.driver.core.ResultSetFuture$ResponseCallback.onSet (ResultSetFuture.java:119) | |
| com.datastax.driver.core.RequestHandler.setFinalResult (RequestHandler.java:202) | |
| com.datastax.driver.core.RequestHandler.onSet (RequestHandler.java:331) | |
| com.datastax.driver.core.Connection$Dispatcher.messageReceived (Connection.java:484) | |
| org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream (SimpleChannelUpstreamHandler.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:296) | |
| org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream (OneToOneDecoder.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream (OneToOneDecoder.java:68) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:296) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived (FrameDecoder.java:462) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode (FrameDecoder.java:443) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived (FrameDecoder.java:303) | |
| org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream (SimpleChannelUpstreamHandler.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:555) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:268) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:255) | |
| org.jboss.netty.channel.socket.nio.NioWorker.read (NioWorker.java:88) | |
| org.jboss.netty.channel.socket.nio.AbstractNioWorker.process (AbstractNioWorker.java:107) | |
| org.jboss.netty.channel.socket.nio.AbstractNioSelector.run (AbstractNioSelector.java:312) | |
| org.jboss.netty.channel.socket.nio.AbstractNioWorker.run (AbstractNioWorker.java:88) | |
| org.jboss.netty.channel.socket.nio.NioWorker.run (NioWorker.java:178) | |
| org.jboss.netty.util.ThreadRenamingRunnable.run (ThreadRenamingRunnable.java:108) | |
| org.jboss.netty.util.internal.DeadLockProofWorker$1.run (DeadLockProofWorker.java:42) | |
| java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1145) | |
| java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:615) | |
| java.lang.Thread.run (Thread.java:722) | |
| 187 :error 10026 ████████████████████▏ | |
| Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| at com.datastax.driver.core.exceptions.WriteTimeoutException.copy (WriteTimeoutException.java:54) | |
| com.datastax.driver.core.ResultSetFuture.extractCauseFromExecutionException (ResultSetFuture.java:214) | |
| com.datastax.driver.core.ResultSetFuture.getUninterruptibly (ResultSetFuture.java:169) | |
| clojurewerkz.cassaforte.client$execute.doInvoke (client.clj:289) | |
| clojure.lang.RestFn.invoke (RestFn.java:457) | |
| jepsen.cassandra$isolation_app$reify__2559$fn__2560.invoke (cassandra.clj:186) | |
| jepsen.cassandra$isolation_app$reify__2559.add (cassandra.clj:177) | |
| jepsen.set_app$eval2420$fn__2443$G__2410__2446.invoke (set_app.clj:11) | |
| jepsen.set_app$eval2420$fn__2443$G__2409__2450.invoke (set_app.clj:11) | |
| clojure.lang.AFn.applyToHelper (AFn.java:163) | |
| clojure.lang.AFn.applyTo (AFn.java:151) | |
| clojure.core$apply.invoke (core.clj:619) | |
| clojure.core$partial$fn__4190.doInvoke (core.clj:2396) | |
| clojure.lang.RestFn.invoke (RestFn.java:408) | |
| jepsen.load$wrap_catch$catcher__2389.invoke (load.clj:110) | |
| jepsen.load$wrap_latency$measure_latency__2386.invoke (load.clj:100) | |
| jepsen.load$wrap_record_req$record_req__2398.invoke (load.clj:129) | |
| jepsen.console$wrap_ordered_log$logger__2346.invoke (console.clj:108) | |
| jepsen.load$map_fixed_rate$boss__2379$fn__2381.invoke (load.clj:65) | |
| clojure.lang.AFn.run (AFn.java:24) | |
| java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1145) | |
| java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:615) | |
| java.lang.Thread.run (Thread.java:722) | |
| Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| at com.datastax.driver.core.ResultSetFuture.convertException (ResultSetFuture.java:243) | |
| com.datastax.driver.core.ResultSetFuture$ResponseCallback.onSet (ResultSetFuture.java:119) | |
| com.datastax.driver.core.RequestHandler.setFinalResult (RequestHandler.java:202) | |
| com.datastax.driver.core.RequestHandler.onSet (RequestHandler.java:331) | |
| com.datastax.driver.core.Connection$Dispatcher.messageReceived (Connection.java:484) | |
| org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream (SimpleChannelUpstreamHandler.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:296) | |
| org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream (OneToOneDecoder.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream (OneToOneDecoder.java:68) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:296) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived (FrameDecoder.java:462) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode (FrameDecoder.java:443) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived (FrameDecoder.java:303) | |
| org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream (SimpleChannelUpstreamHandler.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:555) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:268) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:255) | |
| org.jboss.netty.channel.socket.nio.NioWorker.read (NioWorker.java:88) | |
| org.jboss.netty.channel.socket.nio.AbstractNioWorker.process (AbstractNioWorker.java:107) | |
| org.jboss.netty.channel.socket.nio.AbstractNioSelector.run (AbstractNioSelector.java:312) | |
| org.jboss.netty.channel.socket.nio.AbstractNioWorker.run (AbstractNioWorker.java:88) | |
| org.jboss.netty.channel.socket.nio.NioWorker.run (NioWorker.java:178) | |
| org.jboss.netty.util.ThreadRenamingRunnable.run (ThreadRenamingRunnable.java:108) | |
| org.jboss.netty.util.internal.DeadLockProofWorker$1.run (DeadLockProofWorker.java:42) | |
| java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1145) | |
| java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:615) | |
| java.lang.Thread.run (Thread.java:722) | |
| 188 :ok 21 ▏ | |
| 189 :error 10024 ████████████████████▏ | |
| Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| at com.datastax.driver.core.exceptions.WriteTimeoutException.copy (WriteTimeoutException.java:54) | |
| com.datastax.driver.core.ResultSetFuture.extractCauseFromExecutionException (ResultSetFuture.java:214) | |
| com.datastax.driver.core.ResultSetFuture.getUninterruptibly (ResultSetFuture.java:169) | |
| clojurewerkz.cassaforte.client$execute.doInvoke (client.clj:289) | |
| clojure.lang.RestFn.invoke (RestFn.java:457) | |
| jepsen.cassandra$isolation_app$reify__2559$fn__2560.invoke (cassandra.clj:186) | |
| jepsen.cassandra$isolation_app$reify__2559.add (cassandra.clj:177) | |
| jepsen.set_app$eval2420$fn__2443$G__2410__2446.invoke (set_app.clj:11) | |
| jepsen.set_app$eval2420$fn__2443$G__2409__2450.invoke (set_app.clj:11) | |
| clojure.lang.AFn.applyToHelper (AFn.java:163) | |
| clojure.lang.AFn.applyTo (AFn.java:151) | |
| clojure.core$apply.invoke (core.clj:619) | |
| clojure.core$partial$fn__4190.doInvoke (core.clj:2396) | |
| clojure.lang.RestFn.invoke (RestFn.java:408) | |
| jepsen.load$wrap_catch$catcher__2389.invoke (load.clj:110) | |
| jepsen.load$wrap_latency$measure_latency__2386.invoke (load.clj:100) | |
| jepsen.load$wrap_record_req$record_req__2398.invoke (load.clj:129) | |
| jepsen.console$wrap_ordered_log$logger__2346.invoke (console.clj:108) | |
| jepsen.load$map_fixed_rate$boss__2379$fn__2381.invoke (load.clj:65) | |
| clojure.lang.AFn.run (AFn.java:24) | |
| java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1145) | |
| java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:615) | |
| java.lang.Thread.run (Thread.java:722) | |
| Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| at com.datastax.driver.core.ResultSetFuture.convertException (ResultSetFuture.java:243) | |
| com.datastax.driver.core.ResultSetFuture$ResponseCallback.onSet (ResultSetFuture.java:119) | |
| com.datastax.driver.core.RequestHandler.setFinalResult (RequestHandler.java:202) | |
| com.datastax.driver.core.RequestHandler.onSet (RequestHandler.java:331) | |
| com.datastax.driver.core.Connection$Dispatcher.messageReceived (Connection.java:484) | |
| org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream (SimpleChannelUpstreamHandler.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:296) | |
| org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream (OneToOneDecoder.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream (OneToOneDecoder.java:68) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:296) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived (FrameDecoder.java:462) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode (FrameDecoder.java:443) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived (FrameDecoder.java:303) | |
| org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream (SimpleChannelUpstreamHandler.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:555) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:268) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:255) | |
| org.jboss.netty.channel.socket.nio.NioWorker.read (NioWorker.java:88) | |
| org.jboss.netty.channel.socket.nio.AbstractNioWorker.process (AbstractNioWorker.java:107) | |
| org.jboss.netty.channel.socket.nio.AbstractNioSelector.run (AbstractNioSelector.java:312) | |
| org.jboss.netty.channel.socket.nio.AbstractNioWorker.run (AbstractNioWorker.java:88) | |
| org.jboss.netty.channel.socket.nio.NioWorker.run (NioWorker.java:178) | |
| org.jboss.netty.util.ThreadRenamingRunnable.run (ThreadRenamingRunnable.java:108) | |
| org.jboss.netty.util.internal.DeadLockProofWorker$1.run (DeadLockProofWorker.java:42) | |
| java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1145) | |
| java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:615) | |
| java.lang.Thread.run (Thread.java:722) | |
| 190 :error 10024 ████████████████████▏ | |
| Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| at com.datastax.driver.core.exceptions.WriteTimeoutException.copy (WriteTimeoutException.java:54) | |
| com.datastax.driver.core.ResultSetFuture.extractCauseFromExecutionException (ResultSetFuture.java:214) | |
| com.datastax.driver.core.ResultSetFuture.getUninterruptibly (ResultSetFuture.java:169) | |
| clojurewerkz.cassaforte.client$execute.doInvoke (client.clj:289) | |
| clojure.lang.RestFn.invoke (RestFn.java:457) | |
| jepsen.cassandra$isolation_app$reify__2559$fn__2560.invoke (cassandra.clj:186) | |
| jepsen.cassandra$isolation_app$reify__2559.add (cassandra.clj:177) | |
| jepsen.set_app$eval2420$fn__2443$G__2410__2446.invoke (set_app.clj:11) | |
| jepsen.set_app$eval2420$fn__2443$G__2409__2450.invoke (set_app.clj:11) | |
| clojure.lang.AFn.applyToHelper (AFn.java:163) | |
| clojure.lang.AFn.applyTo (AFn.java:151) | |
| clojure.core$apply.invoke (core.clj:619) | |
| clojure.core$partial$fn__4190.doInvoke (core.clj:2396) | |
| clojure.lang.RestFn.invoke (RestFn.java:408) | |
| jepsen.load$wrap_catch$catcher__2389.invoke (load.clj:110) | |
| jepsen.load$wrap_latency$measure_latency__2386.invoke (load.clj:100) | |
| jepsen.load$wrap_record_req$record_req__2398.invoke (load.clj:129) | |
| jepsen.console$wrap_ordered_log$logger__2346.invoke (console.clj:108) | |
| jepsen.load$map_fixed_rate$boss__2379$fn__2381.invoke (load.clj:65) | |
| clojure.lang.AFn.run (AFn.java:24) | |
| java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1145) | |
| java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:615) | |
| java.lang.Thread.run (Thread.java:722) | |
| Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| at com.datastax.driver.core.ResultSetFuture.convertException (ResultSetFuture.java:243) | |
| com.datastax.driver.core.ResultSetFuture$ResponseCallback.onSet (ResultSetFuture.java:119) | |
| com.datastax.driver.core.RequestHandler.setFinalResult (RequestHandler.java:202) | |
| com.datastax.driver.core.RequestHandler.onSet (RequestHandler.java:331) | |
| com.datastax.driver.core.Connection$Dispatcher.messageReceived (Connection.java:484) | |
| org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream (SimpleChannelUpstreamHandler.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:296) | |
| org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream (OneToOneDecoder.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream (OneToOneDecoder.java:68) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:296) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived (FrameDecoder.java:462) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode (FrameDecoder.java:443) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived (FrameDecoder.java:303) | |
| org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream (SimpleChannelUpstreamHandler.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:555) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:268) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:255) | |
| org.jboss.netty.channel.socket.nio.NioWorker.read (NioWorker.java:88) | |
| org.jboss.netty.channel.socket.nio.AbstractNioWorker.process (AbstractNioWorker.java:107) | |
| org.jboss.netty.channel.socket.nio.AbstractNioSelector.run (AbstractNioSelector.java:312) | |
| org.jboss.netty.channel.socket.nio.AbstractNioWorker.run (AbstractNioWorker.java:88) | |
| org.jboss.netty.channel.socket.nio.NioWorker.run (NioWorker.java:178) | |
| org.jboss.netty.util.ThreadRenamingRunnable.run (ThreadRenamingRunnable.java:108) | |
| org.jboss.netty.util.internal.DeadLockProofWorker$1.run (DeadLockProofWorker.java:42) | |
| java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1145) | |
| java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:615) | |
| java.lang.Thread.run (Thread.java:722) | |
| 191 :ok 19 ▏ | |
| 192 :ok 17 ▏ | |
| 193 :ok 20 ▏ | |
| 194 :ok 18 ▏ | |
| 195 :ok 21 ▏ | |
| 196 :ok 21 ▏ | |
| 197 :ok 21 ▏ | |
| 198 :ok 19 ▏ | |
| 199 :ok 18 ▏ | |
| 200 :ok 22 ▏ | |
| 201 :ok 27 ▏ | |
| 202 :ok 26 ▏ | |
| 203 :ok 24 ▏ | |
| 204 :ok 23 ▏ | |
| 205 :ok 16 ▏ | |
| 206 :ok 20 ▏ | |
| 207 :ok 22 ▏ | |
| 208 :ok 19 ▏ | |
| 209 :ok 21 ▏ | |
| 210 :ok 17 ▏ | |
| 211 :ok 14 ▏ | |
| 212 :ok 14 ▏ | |
| 213 :ok 14 ▏ | |
| 214 :ok 14 ▏ | |
| 215 :ok 40 ▏ | |
| 216 :ok 38 ▏ | |
| 217 :ok 34 ▏ | |
| 218 :ok 38 ▏ | |
| 219 :ok 38 ▏ | |
| 220 :error 10027 ████████████████████▏ | |
| Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| at com.datastax.driver.core.exceptions.WriteTimeoutException.copy (WriteTimeoutException.java:54) | |
| com.datastax.driver.core.ResultSetFuture.extractCauseFromExecutionException (ResultSetFuture.java:214) | |
| com.datastax.driver.core.ResultSetFuture.getUninterruptibly (ResultSetFuture.java:169) | |
| clojurewerkz.cassaforte.client$execute.doInvoke (client.clj:289) | |
| clojure.lang.RestFn.invoke (RestFn.java:457) | |
| jepsen.cassandra$isolation_app$reify__2559$fn__2560.invoke (cassandra.clj:186) | |
| jepsen.cassandra$isolation_app$reify__2559.add (cassandra.clj:177) | |
| jepsen.set_app$eval2420$fn__2443$G__2410__2446.invoke (set_app.clj:11) | |
| jepsen.set_app$eval2420$fn__2443$G__2409__2450.invoke (set_app.clj:11) | |
| clojure.lang.AFn.applyToHelper (AFn.java:163) | |
| clojure.lang.AFn.applyTo (AFn.java:151) | |
| clojure.core$apply.invoke (core.clj:619) | |
| clojure.core$partial$fn__4190.doInvoke (core.clj:2396) | |
| clojure.lang.RestFn.invoke (RestFn.java:408) | |
| jepsen.load$wrap_catch$catcher__2389.invoke (load.clj:110) | |
| jepsen.load$wrap_latency$measure_latency__2386.invoke (load.clj:100) | |
| jepsen.load$wrap_record_req$record_req__2398.invoke (load.clj:129) | |
| jepsen.console$wrap_ordered_log$logger__2346.invoke (console.clj:108) | |
| jepsen.load$map_fixed_rate$boss__2379$fn__2381.invoke (load.clj:65) | |
| clojure.lang.AFn.run (AFn.java:24) | |
| java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1145) | |
| java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:615) | |
| java.lang.Thread.run (Thread.java:722) | |
| Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| at com.datastax.driver.core.ResultSetFuture.convertException (ResultSetFuture.java:243) | |
| com.datastax.driver.core.ResultSetFuture$ResponseCallback.onSet (ResultSetFuture.java:119) | |
| com.datastax.driver.core.RequestHandler.setFinalResult (RequestHandler.java:202) | |
| com.datastax.driver.core.RequestHandler.onSet (RequestHandler.java:331) | |
| com.datastax.driver.core.Connection$Dispatcher.messageReceived (Connection.java:484) | |
| org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream (SimpleChannelUpstreamHandler.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:296) | |
| org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream (OneToOneDecoder.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream (OneToOneDecoder.java:68) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:296) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived (FrameDecoder.java:462) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode (FrameDecoder.java:443) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived (FrameDecoder.java:303) | |
| org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream (SimpleChannelUpstreamHandler.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:555) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:268) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:255) | |
| org.jboss.netty.channel.socket.nio.NioWorker.read (NioWorker.java:88) | |
| org.jboss.netty.channel.socket.nio.AbstractNioWorker.process (AbstractNioWorker.java:107) | |
| org.jboss.netty.channel.socket.nio.AbstractNioSelector.run (AbstractNioSelector.java:312) | |
| org.jboss.netty.channel.socket.nio.AbstractNioWorker.run (AbstractNioWorker.java:88) | |
| org.jboss.netty.channel.socket.nio.NioWorker.run (NioWorker.java:178) | |
| org.jboss.netty.util.ThreadRenamingRunnable.run (ThreadRenamingRunnable.java:108) | |
| org.jboss.netty.util.internal.DeadLockProofWorker$1.run (DeadLockProofWorker.java:42) | |
| java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1145) | |
| java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:615) | |
| java.lang.Thread.run (Thread.java:722) | |
| 221 :ok 21 ▏ | |
| 222 :ok 20 ▏ | |
| 223 :ok 16 ▏ | |
| 224 :ok 19 ▏ | |
| 225 :ok 20 ▏ | |
| 226 :ok 22 ▏ | |
| 227 :ok 25 ▏ | |
| 228 :ok 19 ▏ | |
| 229 :ok 24 ▏ | |
| 230 :ok 27 ▏ | |
| 231 :ok 23 ▏ | |
| 232 :ok 25 ▏ | |
| 233 :ok 23 ▏ | |
| 234 :ok 22 ▏ | |
| 235 :error 10022 ████████████████████▏ | |
| Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| at com.datastax.driver.core.exceptions.WriteTimeoutException.copy (WriteTimeoutException.java:54) | |
| com.datastax.driver.core.ResultSetFuture.extractCauseFromExecutionException (ResultSetFuture.java:214) | |
| com.datastax.driver.core.ResultSetFuture.getUninterruptibly (ResultSetFuture.java:169) | |
| clojurewerkz.cassaforte.client$execute.doInvoke (client.clj:289) | |
| clojure.lang.RestFn.invoke (RestFn.java:457) | |
| jepsen.cassandra$isolation_app$reify__2559$fn__2560.invoke (cassandra.clj:186) | |
| jepsen.cassandra$isolation_app$reify__2559.add (cassandra.clj:177) | |
| jepsen.set_app$eval2420$fn__2443$G__2410__2446.invoke (set_app.clj:11) | |
| jepsen.set_app$eval2420$fn__2443$G__2409__2450.invoke (set_app.clj:11) | |
| clojure.lang.AFn.applyToHelper (AFn.java:163) | |
| clojure.lang.AFn.applyTo (AFn.java:151) | |
| clojure.core$apply.invoke (core.clj:619) | |
| clojure.core$partial$fn__4190.doInvoke (core.clj:2396) | |
| clojure.lang.RestFn.invoke (RestFn.java:408) | |
| jepsen.load$wrap_catch$catcher__2389.invoke (load.clj:110) | |
| jepsen.load$wrap_latency$measure_latency__2386.invoke (load.clj:100) | |
| jepsen.load$wrap_record_req$record_req__2398.invoke (load.clj:129) | |
| jepsen.console$wrap_ordered_log$logger__2346.invoke (console.clj:108) | |
| jepsen.load$map_fixed_rate$boss__2379$fn__2381.invoke (load.clj:65) | |
| clojure.lang.AFn.run (AFn.java:24) | |
| java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1145) | |
| java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:615) | |
| java.lang.Thread.run (Thread.java:722) | |
| Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| at com.datastax.driver.core.ResultSetFuture.convertException (ResultSetFuture.java:243) | |
| com.datastax.driver.core.ResultSetFuture$ResponseCallback.onSet (ResultSetFuture.java:119) | |
| com.datastax.driver.core.RequestHandler.setFinalResult (RequestHandler.java:202) | |
| com.datastax.driver.core.RequestHandler.onSet (RequestHandler.java:331) | |
| com.datastax.driver.core.Connection$Dispatcher.messageReceived (Connection.java:484) | |
| org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream (SimpleChannelUpstreamHandler.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:296) | |
| org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream (OneToOneDecoder.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream (OneToOneDecoder.java:68) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:296) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived (FrameDecoder.java:462) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode (FrameDecoder.java:443) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived (FrameDecoder.java:303) | |
| org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream (SimpleChannelUpstreamHandler.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:555) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:268) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:255) | |
| org.jboss.netty.channel.socket.nio.NioWorker.read (NioWorker.java:88) | |
| org.jboss.netty.channel.socket.nio.AbstractNioWorker.process (AbstractNioWorker.java:107) | |
| org.jboss.netty.channel.socket.nio.AbstractNioSelector.run (AbstractNioSelector.java:312) | |
| org.jboss.netty.channel.socket.nio.AbstractNioWorker.run (AbstractNioWorker.java:88) | |
| org.jboss.netty.channel.socket.nio.NioWorker.run (NioWorker.java:178) | |
| org.jboss.netty.util.ThreadRenamingRunnable.run (ThreadRenamingRunnable.java:108) | |
| org.jboss.netty.util.internal.DeadLockProofWorker$1.run (DeadLockProofWorker.java:42) | |
| java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1145) | |
| java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:615) | |
| java.lang.Thread.run (Thread.java:722) | |
| 236 :ok 18 ▏ | |
| 237 :ok 16 ▏ | |
| 238 :ok 16 ▏ | |
| 239 :ok 19 ▏ | |
| 240 :ok 20 ▏ | |
| 241 :ok 26 ▏ | |
| 242 :ok 28 ▏ | |
| 243 :ok 20 ▏ | |
| 244 :ok 23 ▏ | |
| 245 :error 10022 ████████████████████▏ | |
| Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| at com.datastax.driver.core.exceptions.WriteTimeoutException.copy (WriteTimeoutException.java:54) | |
| com.datastax.driver.core.ResultSetFuture.extractCauseFromExecutionException (ResultSetFuture.java:214) | |
| com.datastax.driver.core.ResultSetFuture.getUninterruptibly (ResultSetFuture.java:169) | |
| clojurewerkz.cassaforte.client$execute.doInvoke (client.clj:289) | |
| clojure.lang.RestFn.invoke (RestFn.java:457) | |
| jepsen.cassandra$isolation_app$reify__2559$fn__2560.invoke (cassandra.clj:186) | |
| jepsen.cassandra$isolation_app$reify__2559.add (cassandra.clj:177) | |
| jepsen.set_app$eval2420$fn__2443$G__2410__2446.invoke (set_app.clj:11) | |
| jepsen.set_app$eval2420$fn__2443$G__2409__2450.invoke (set_app.clj:11) | |
| clojure.lang.AFn.applyToHelper (AFn.java:163) | |
| clojure.lang.AFn.applyTo (AFn.java:151) | |
| clojure.core$apply.invoke (core.clj:619) | |
| clojure.core$partial$fn__4190.doInvoke (core.clj:2396) | |
| clojure.lang.RestFn.invoke (RestFn.java:408) | |
| jepsen.load$wrap_catch$catcher__2389.invoke (load.clj:110) | |
| jepsen.load$wrap_latency$measure_latency__2386.invoke (load.clj:100) | |
| jepsen.load$wrap_record_req$record_req__2398.invoke (load.clj:129) | |
| jepsen.console$wrap_ordered_log$logger__2346.invoke (console.clj:108) | |
| jepsen.load$map_fixed_rate$boss__2379$fn__2381.invoke (load.clj:65) | |
| clojure.lang.AFn.run (AFn.java:24) | |
| java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1145) | |
| java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:615) | |
| java.lang.Thread.run (Thread.java:722) | |
| Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| at com.datastax.driver.core.ResultSetFuture.convertException (ResultSetFuture.java:243) | |
| com.datastax.driver.core.ResultSetFuture$ResponseCallback.onSet (ResultSetFuture.java:119) | |
| com.datastax.driver.core.RequestHandler.setFinalResult (RequestHandler.java:202) | |
| com.datastax.driver.core.RequestHandler.onSet (RequestHandler.java:331) | |
| com.datastax.driver.core.Connection$Dispatcher.messageReceived (Connection.java:484) | |
| org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream (SimpleChannelUpstreamHandler.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:296) | |
| org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream (OneToOneDecoder.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream (OneToOneDecoder.java:68) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:296) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived (FrameDecoder.java:462) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode (FrameDecoder.java:443) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived (FrameDecoder.java:303) | |
| org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream (SimpleChannelUpstreamHandler.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:555) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:268) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:255) | |
| org.jboss.netty.channel.socket.nio.NioWorker.read (NioWorker.java:88) | |
| org.jboss.netty.channel.socket.nio.AbstractNioWorker.process (AbstractNioWorker.java:107) | |
| org.jboss.netty.channel.socket.nio.AbstractNioSelector.run (AbstractNioSelector.java:312) | |
| org.jboss.netty.channel.socket.nio.AbstractNioWorker.run (AbstractNioWorker.java:88) | |
| org.jboss.netty.channel.socket.nio.NioWorker.run (NioWorker.java:178) | |
| org.jboss.netty.util.ThreadRenamingRunnable.run (ThreadRenamingRunnable.java:108) | |
| org.jboss.netty.util.internal.DeadLockProofWorker$1.run (DeadLockProofWorker.java:42) | |
| java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1145) | |
| java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:615) | |
| java.lang.Thread.run (Thread.java:722) | |
| 246 :error 10033 ████████████████████▏ | |
| Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| at com.datastax.driver.core.exceptions.WriteTimeoutException.copy (WriteTimeoutException.java:54) | |
| com.datastax.driver.core.ResultSetFuture.extractCauseFromExecutionException (ResultSetFuture.java:214) | |
| com.datastax.driver.core.ResultSetFuture.getUninterruptibly (ResultSetFuture.java:169) | |
| clojurewerkz.cassaforte.client$execute.doInvoke (client.clj:289) | |
| clojure.lang.RestFn.invoke (RestFn.java:457) | |
| jepsen.cassandra$isolation_app$reify__2559$fn__2560.invoke (cassandra.clj:186) | |
| jepsen.cassandra$isolation_app$reify__2559.add (cassandra.clj:177) | |
| jepsen.set_app$eval2420$fn__2443$G__2410__2446.invoke (set_app.clj:11) | |
| jepsen.set_app$eval2420$fn__2443$G__2409__2450.invoke (set_app.clj:11) | |
| clojure.lang.AFn.applyToHelper (AFn.java:163) | |
| clojure.lang.AFn.applyTo (AFn.java:151) | |
| clojure.core$apply.invoke (core.clj:619) | |
| clojure.core$partial$fn__4190.doInvoke (core.clj:2396) | |
| clojure.lang.RestFn.invoke (RestFn.java:408) | |
| jepsen.load$wrap_catch$catcher__2389.invoke (load.clj:110) | |
| jepsen.load$wrap_latency$measure_latency__2386.invoke (load.clj:100) | |
| jepsen.load$wrap_record_req$record_req__2398.invoke (load.clj:129) | |
| jepsen.console$wrap_ordered_log$logger__2346.invoke (console.clj:108) | |
| jepsen.load$map_fixed_rate$boss__2379$fn__2381.invoke (load.clj:65) | |
| clojure.lang.AFn.run (AFn.java:24) | |
| java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1145) | |
| java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:615) | |
| java.lang.Thread.run (Thread.java:722) | |
| Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| at com.datastax.driver.core.ResultSetFuture.convertException (ResultSetFuture.java:243) | |
| com.datastax.driver.core.ResultSetFuture$ResponseCallback.onSet (ResultSetFuture.java:119) | |
| com.datastax.driver.core.RequestHandler.setFinalResult (RequestHandler.java:202) | |
| com.datastax.driver.core.RequestHandler.onSet (RequestHandler.java:331) | |
| com.datastax.driver.core.Connection$Dispatcher.messageReceived (Connection.java:484) | |
| org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream (SimpleChannelUpstreamHandler.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:296) | |
| org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream (OneToOneDecoder.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream (OneToOneDecoder.java:68) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:296) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived (FrameDecoder.java:462) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode (FrameDecoder.java:443) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived (FrameDecoder.java:303) | |
| org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream (SimpleChannelUpstreamHandler.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:555) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:268) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:255) | |
| org.jboss.netty.channel.socket.nio.NioWorker.read (NioWorker.java:88) | |
| org.jboss.netty.channel.socket.nio.AbstractNioWorker.process (AbstractNioWorker.java:107) | |
| org.jboss.netty.channel.socket.nio.AbstractNioSelector.run (AbstractNioSelector.java:312) | |
| org.jboss.netty.channel.socket.nio.AbstractNioWorker.run (AbstractNioWorker.java:88) | |
| org.jboss.netty.channel.socket.nio.NioWorker.run (NioWorker.java:178) | |
| org.jboss.netty.util.ThreadRenamingRunnable.run (ThreadRenamingRunnable.java:108) | |
| org.jboss.netty.util.internal.DeadLockProofWorker$1.run (DeadLockProofWorker.java:42) | |
| java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1145) | |
| java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:615) | |
| java.lang.Thread.run (Thread.java:722) | |
| 247 :ok 5083 ██████████▎ | |
| 248 :ok 5211 ██████████▌ | |
| 249 :ok 5210 ██████████▌ | |
| 250 :error 10025 ████████████████████▏ | |
| Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| at com.datastax.driver.core.exceptions.WriteTimeoutException.copy (WriteTimeoutException.java:54) | |
| com.datastax.driver.core.ResultSetFuture.extractCauseFromExecutionException (ResultSetFuture.java:214) | |
| com.datastax.driver.core.ResultSetFuture.getUninterruptibly (ResultSetFuture.java:169) | |
| clojurewerkz.cassaforte.client$execute.doInvoke (client.clj:289) | |
| clojure.lang.RestFn.invoke (RestFn.java:457) | |
| jepsen.cassandra$isolation_app$reify__2559$fn__2560.invoke (cassandra.clj:186) | |
| jepsen.cassandra$isolation_app$reify__2559.add (cassandra.clj:177) | |
| jepsen.set_app$eval2420$fn__2443$G__2410__2446.invoke (set_app.clj:11) | |
| jepsen.set_app$eval2420$fn__2443$G__2409__2450.invoke (set_app.clj:11) | |
| clojure.lang.AFn.applyToHelper (AFn.java:163) | |
| clojure.lang.AFn.applyTo (AFn.java:151) | |
| clojure.core$apply.invoke (core.clj:619) | |
| clojure.core$partial$fn__4190.doInvoke (core.clj:2396) | |
| clojure.lang.RestFn.invoke (RestFn.java:408) | |
| jepsen.load$wrap_catch$catcher__2389.invoke (load.clj:110) | |
| jepsen.load$wrap_latency$measure_latency__2386.invoke (load.clj:100) | |
| jepsen.load$wrap_record_req$record_req__2398.invoke (load.clj:129) | |
| jepsen.console$wrap_ordered_log$logger__2346.invoke (console.clj:108) | |
| jepsen.load$map_fixed_rate$boss__2379$fn__2381.invoke (load.clj:65) | |
| clojure.lang.AFn.run (AFn.java:24) | |
| java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1145) | |
| java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:615) | |
| java.lang.Thread.run (Thread.java:722) | |
| Caused by: com.datastax.driver.core.exceptions.WriteTimeoutException: Cassandra timeout during write query at consistency ANY (1 replica were required but only 0 acknowledged the write) | |
| at com.datastax.driver.core.ResultSetFuture.convertException (ResultSetFuture.java:243) | |
| com.datastax.driver.core.ResultSetFuture$ResponseCallback.onSet (ResultSetFuture.java:119) | |
| com.datastax.driver.core.RequestHandler.setFinalResult (RequestHandler.java:202) | |
| com.datastax.driver.core.RequestHandler.onSet (RequestHandler.java:331) | |
| com.datastax.driver.core.Connection$Dispatcher.messageReceived (Connection.java:484) | |
| org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream (SimpleChannelUpstreamHandler.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:296) | |
| org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream (OneToOneDecoder.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream (OneToOneDecoder.java:68) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream (DefaultChannelPipeline.java:787) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:296) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived (FrameDecoder.java:462) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode (FrameDecoder.java:443) | |
| org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived (FrameDecoder.java:303) | |
| org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream (SimpleChannelUpstreamHandler.java:70) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:560) | |
| org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream (DefaultChannelPipeline.java:555) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:268) | |
| org.jboss.netty.channel.Channels.fireMessageReceived (Channels.java:255) | |
| org.jboss.netty.channel.socket.nio.NioWorker.read (NioWorker.java:88) | |
| org.jboss.netty.channel.socket.nio.AbstractNioWorker.process (AbstractNioWorker.java:107) | |
| org.jboss.netty.channel.socket.nio.AbstractNioSelector.run (AbstractNioSelector.java:312) | |
| org.jboss.netty.channel.socket.nio.AbstractNioWorker.run (AbstractNioWorker.java:88) | |
| org.jboss.netty.channel.socket.nio.NioWorker.run (NioWorker.java:178) | |
| org.jboss.netty.util.ThreadRenamingRunnable.run (ThreadRenamingRunnable.java:108) | |
| org.jboss.netty.util.internal.DeadLockProofWorker$1.run (DeadLockProofWorker.java:42) | |
| java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1145) | |
| java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:615) | |
| java.lang.Thread.run (Thread.java:722) | |
| 251 :ok 15 ▏ | |
| 252 :ok 4585 █████████▎ | |
| 253 :ok 20 ▏ | |
| 254 :ok 17 ▏ | |
| 255 :ok 24 ▏ | |
| 256 :ok 18 ▏ | |
| 257 :ok 16 ▏ | |
| 258 :ok 19 ▏ | |
| 259 :ok 20 ▏ | |
| 260 :ok 26 ▏ | |
| 261 :ok 22 ▏ | |
| 262 :ok 22 ▏ | |
| 263 :ok 24 ▏ | |
| 264 :ok 2728 █████▌ | |
| 265 :ok 19 ▏ | |
| 266 :ok 14 ▏ | |
| 267 :ok 15 ▏ | |
| 268 :ok 3212 ██████▌ | |
| 269 :ok 2221 ████▌ | |
| 270 :ok 23 ▏ | |
| 271 :ok 21 ▏ | |
| 272 :ok 2582 █████▎ | |
| 273 :ok 24 ▏ | |
| 274 :ok 23 ▏ | |
| 275 :ok 22 ▏ | |
| 276 :ok 22 ▏ | |
| 277 :ok 26 ▏ | |
| 278 :ok 29 ▏ | |
| 279 :ok 25 ▏ | |
| 280 :ok 19 ▏ | |
| 281 :ok 18 ▏ | |
| 282 :ok 16 ▏ | |
| 283 :ok 724 █▌ | |
| 284 :ok 18 ▏ | |
| 285 :ok 27 ▏ | |
| 286 :ok 23 ▏ | |
| 287 :ok 25 ▏ | |
| 288 :ok 24 ▏ | |
| 289 :ok 23 ▏ | |
| 290 :ok 29 ▏ | |
| 291 :ok 29 ▏ | |
| 292 :ok 25 ▏ | |
| 293 :ok 32 ▏ | |
| 294 :ok 33 ▏ | |
| 295 :ok 20 ▏ | |
| 296 :ok 19 ▏ | |
| 297 :ok 20 ▏ | |
| 298 :ok 19 ▏ | |
| 299 :ok 22 ▏ | |
| 300 :ok 22 ▏ | |
| 301 :ok 24 ▏ | |
| 302 :ok 24 ▏ | |
| 303 :ok 24 ▏ | |
| 304 :ok 23 ▏ | |
| 305 :ok 25 ▏ | |
| 306 :ok 24 ▏ | |
| 307 :ok 26 ▏ | |
| 308 :ok 23 ▏ | |
| 309 :ok 23 ▏ | |
| 310 :ok 18 ▏ | |
| 311 :ok 16 ▏ | |
| 312 :ok 19 ▏ | |
| 313 :ok 18 ▏ | |
| 314 :ok 20 ▏ | |
| 315 :ok 19 ▏ | |
| 316 :ok 19 ▏ | |
| 317 :ok 19 ▏ | |
| 318 :ok 18 ▏ | |
| 319 :ok 20 ▏ | |
| 320 :ok 29 ▏ | |
| 321 :ok 29 ▏ | |
| 322 :ok 26 ▏ | |
| 323 :ok 20 ▏ | |
| 324 :ok 27 ▏ | |
| 325 :ok 20 ▏ | |
| 326 :ok 19 ▏ | |
| 327 :ok 20 ▏ | |
| 328 :ok 17 ▏ | |
| 329 :ok 17 ▏ | |
| 330 :ok 18 ▏ | |
| 331 :ok 22 ▏ | |
| 332 :ok 22 ▏ | |
| 333 :ok 18 ▏ | |
| 334 :ok 23 ▏ | |
| 335 :ok 29 ▏ | |
| 336 :ok 29 ▏ | |
| 337 :ok 32 ▏ | |
| 338 :ok 25 ▏ | |
| 339 :ok 24 ▏ | |
| 340 :ok 19 ▏ | |
| 341 :ok 19 ▏ | |
| 342 :ok 21 ▏ | |
| 343 :ok 17 ▏ | |
| 344 :ok 19 ▏ | |
| 345 :ok 23 ▏ | |
| 346 :ok 24 ▏ | |
| 347 :ok 26 ▏ | |
| 348 :ok 21 ▏ | |
| 349 :ok 19 ▏ | |
| 350 :ok 28 ▏ | |
| 351 :ok 34 ▏ | |
| 352 :ok 29 ▏ | |
| 353 :ok 24 ▏ | |
| 354 :ok 35 ▏ | |
| 355 :ok 22 ▏ | |
| 356 :ok 22 ▏ | |
| 357 :ok 21 ▏ | |
| 358 :ok 17 ▏ | |
| 359 :ok 19 ▏ | |
| 360 :ok 27 ▏ | |
| 361 :ok 28 ▏ | |
| 362 :ok 26 ▏ | |
| 363 :ok 21 ▏ | |
| 364 :ok 23 ▏ | |
| 365 :ok 19 ▏ | |
| 366 :ok 25 ▏ | |
| 367 :ok 21 ▏ | |
| 368 :ok 20 ▏ | |
| 369 :ok 22 ▏ | |
| 370 :ok 25 ▏ | |
| 371 :ok 30 ▏ | |
| 372 :ok 30 ▏ | |
| 373 :ok 28 ▏ | |
| 374 :ok 25 ▏ | |
| 375 :ok 21 ▏ | |
| 376 :ok 19 ▏ | |
| 377 :ok 25 ▏ | |
| 378 :ok 22 ▏ | |
| 379 :ok 21 ▏ | |
| 380 :ok 32 ▏ | |
| 381 :ok 31 ▏ | |
| 382 :ok 32 ▏ | |
| 383 :ok 31 ▏ | |
| 384 :ok 27 ▏ | |
| 385 :ok 20 ▏ | |
| 386 :ok 16 ▏ | |
| 387 :ok 17 ▏ | |
| 388 :ok 15 ▏ | |
| 389 :ok 15 ▏ | |
| 390 :ok 19 ▏ | |
| 391 :ok 24 ▏ | |
| 392 :ok 24 ▏ | |
| 393 :ok 19 ▏ | |
| 394 :ok 18 ▏ | |
| 395 :ok 23 ▏ | |
| 396 :ok 23 ▏ | |
| 397 :ok 25 ▏ | |
| 398 :ok 25 ▏ | |
| 399 :ok 24 ▏ | |
| 400 :ok 17 ▏ | |
| 401 :ok 20 ▏ | |
| 402 :ok 17 ▏ | |
| 403 :ok 18 ▏ | |
| 404 :ok 14 ▏ | |
| 405 :ok 18 ▏ | |
| 406 :ok 19 ▏ | |
| 407 :ok 20 ▏ | |
| 408 :ok 23 ▏ | |
| 409 :ok 21 ▏ | |
| 410 :ok 18 ▏ | |
| 411 :ok 20 ▏ | |
| 412 :ok 24 ▏ | |
| 413 :ok 23 ▏ | |
| 414 :ok 24 ▏ | |
| 415 :ok 18 ▏ | |
| 416 :ok 19 ▏ | |
| 417 :ok 15 ▏ | |
| 418 :ok 16 ▏ | |
| 419 :ok 18 ▏ | |
| 420 :ok 19 ▏ | |
| 421 :ok 22 ▏ | |
| 422 :ok 22 ▏ | |
| 423 :ok 17 ▏ | |
| 424 :ok 23 ▏ | |
| 425 :ok 24 ▏ | |
| 426 :ok 22 ▏ | |
| 427 :ok 25 ▏ | |
| 428 :ok 21 ▏ | |
| 429 :ok 21 ▏ | |
| 430 :ok 18 ▏ | |
| 431 :ok 17 ▏ | |
| 432 :ok 19 ▏ | |
| 433 :ok 14 ▏ | |
| 434 :ok 20 ▏ | |
| 435 :ok 21 ▏ | |
| 436 :ok 20 ▏ | |
| 437 :ok 18 ▏ | |
| 438 :ok 19 ▏ | |
| 439 :ok 18 ▏ | |
| 440 :ok 61 ▏ | |
| 441 :ok 21 ▏ | |
| 442 :ok 60 ▏ | |
| 443 :ok 63 ▎ | |
| 444 :ok 62 ▏ | |
| 445 :ok 23 ▏ | |
| 446 :ok 25 ▏ | |
| 447 :ok 24 ▏ | |
| 448 :ok 22 ▏ | |
| 449 :ok 20 ▏ | |
| 450 :ok 18 ▏ | |
| 451 :ok 18 ▏ | |
| 452 :ok 24 ▏ | |
| 453 :ok 21 ▏ | |
| 454 :ok 19 ▏ | |
| 455 :ok 24 ▏ | |
| 456 :ok 25 ▏ | |
| 457 :ok 27 ▏ | |
| 458 :ok 22 ▏ | |
| 459 :ok 22 ▏ | |
| 460 :ok 15 ▏ | |
| 461 :ok 14 ▏ | |
| 462 :ok 17 ▏ | |
| 463 :ok 18 ▏ | |
| 464 :ok 21 ▏ | |
| 465 :ok 19 ▏ | |
| 466 :ok 21 ▏ | |
| 467 :ok 22 ▏ | |
| 468 :ok 19 ▏ | |
| 469 :ok 22 ▏ | |
| 470 :ok 21 ▏ | |
| 471 :ok 19 ▏ | |
| 472 :ok 26 ▏ | |
| 473 :ok 21 ▏ | |
| 474 :ok 22 ▏ | |
| 475 :ok 18 ▏ | |
| 476 :ok 17 ▏ | |
| 477 :ok 20 ▏ | |
| 478 :ok 20 ▏ | |
| 479 :ok 21 ▏ | |
| 480 :ok 19 ▏ | |
| 481 :ok 23 ▏ | |
| 482 :ok 21 ▏ | |
| 483 :ok 19 ▏ | |
| 484 :ok 23 ▏ | |
| 485 :ok 28 ▏ | |
| 486 :ok 23 ▏ | |
| 487 :ok 29 ▏ | |
| 488 :ok 22 ▏ | |
| 489 :ok 25 ▏ | |
| 490 :ok 14 ▏ | |
| 491 :ok 20 ▏ | |
| 492 :ok 20 ▏ | |
| 493 :ok 14 ▏ | |
| 494 :ok 19 ▏ | |
| 495 :ok 22 ▏ | |
| 496 :ok 21 ▏ | |
| 497 :ok 19 ▏ | |
| 498 :ok 17 ▏ | |
| 499 :ok 21 ▏ | |
| 500 :ok 23 ▏ | |
| 501 :ok 21 ▏ | |
| 502 :ok 22 ▏ | |
| 503 :ok 25 ▏ | |
| 504 :ok 22 ▏ | |
| 505 :ok 20 ▏ | |
| 506 :ok 20 ▏ | |
| 507 :ok 21 ▏ | |
| 508 :ok 17 ▏ | |
| 509 :ok 19 ▏ | |
| 510 :ok 29 ▏ | |
| 511 :ok 35 ▏ | |
| 512 :ok 26 ▏ | |
| 513 :ok 34 ▏ | |
| 514 :ok 60 ▏ | |
| Partition healed. | |
| 515 :ok 23 ▏ | |
| 516 :ok 22 ▏ | |
| 517 :ok 27 ▏ | |
| 518 :ok 25 ▏ | |
| 519 :ok 21 ▏ | |
| 520 :ok 18 ▏ | |
| 521 :ok 19 ▏ | |
| 522 :ok 19 ▏ | |
| 523 :ok 13 ▏ | |
| 524 :ok 18 ▏ | |
| 525 :ok 19 ▏ | |
| 526 :ok 25 ▏ | |
| 527 :ok 24 ▏ | |
| 528 :ok 22 ▏ | |
| 529 :ok 23 ▏ | |
| 530 :ok 16 ▏ | |
| 531 :ok 18 ▏ | |
| 532 :ok 18 ▏ | |
| 533 :ok 14 ▏ | |
| 534 :ok 17 ▏ | |
| 535 :ok 15 ▏ | |
| 536 :ok 13 ▏ | |
| 537 :ok 19 ▏ | |
| 538 :ok 15 ▏ | |
| 539 :ok 13 ▏ | |
| 540 :ok 22 ▏ | |
| 541 :ok 18 ▏ | |
| 542 :ok 20 ▏ | |
| 543 :ok 22 ▏ | |
| 544 :ok 15 ▏ | |
| 545 :ok 21 ▏ | |
| 546 :ok 24 ▏ | |
| 547 :ok 24 ▏ | |
| 548 :ok 23 ▏ | |
| 549 :ok 21 ▏ | |
| 550 :ok 17 ▏ | |
| 551 :ok 20 ▏ | |
| 552 :ok 19 ▏ | |
| 553 :ok 18 ▏ | |
| 554 :ok 19 ▏ | |
| 555 :ok 19 ▏ | |
| 556 :ok 19 ▏ | |
| 557 :ok 23 ▏ | |
| 558 :ok 20 ▏ | |
| 559 :ok 19 ▏ | |
| 560 :ok 25 ▏ | |
| 561 :ok 24 ▏ | |
| 562 :ok 22 ▏ | |
| 563 :ok 22 ▏ | |
| 564 :ok 23 ▏ | |
| 565 :ok 18 ▏ | |
| 566 :ok 16 ▏ | |
| 567 :ok 22 ▏ | |
| 568 :ok 16 ▏ | |
| 569 :ok 20 ▏ | |
| 570 :ok 23 ▏ | |
| 571 :ok 24 ▏ | |
| 572 :ok 19 ▏ | |
| 573 :ok 23 ▏ | |
| 574 :ok 21 ▏ | |
| 575 :ok 24 ▏ | |
| 576 :ok 20 ▏ | |
| 577 :ok 25 ▏ | |
| 578 :ok 25 ▏ | |
| 579 :ok 25 ▏ | |
| 580 :ok 21 ▏ | |
| 581 :ok 28 ▏ | |
| 582 :ok 24 ▏ | |
| 583 :ok 22 ▏ | |
| 584 :ok 20 ▏ | |
| 585 :ok 18 ▏ | |
| 586 :ok 21 ▏ | |
| 587 :ok 21 ▏ | |
| 588 :ok 21 ▏ | |
| 589 :ok 22 ▏ | |
| 590 :ok 18 ▏ | |
| 591 :ok 23 ▏ | |
| 592 :ok 26 ▏ | |
| 593 :ok 25 ▏ | |
| 594 :ok 22 ▏ | |
| 595 :ok 18 ▏ | |
| 596 :ok 17 ▏ | |
| 597 :ok 16 ▏ | |
| 598 :ok 19 ▏ | |
| 599 :ok 18 ▏ | |
| 600 :ok 22 ▏ | |
| 601 :ok 23 ▏ | |
| 602 :ok 21 ▏ | |
| 603 :ok 18 ▏ | |
| 604 :ok 22 ▏ | |
| 605 :ok 19 ▏ | |
| 606 :ok 20 ▏ | |
| 607 :ok 24 ▏ | |
| 608 :ok 22 ▏ | |
| 609 :ok 22 ▏ | |
| 610 :ok 28 ▏ | |
| 611 :ok 27 ▏ | |
| 612 :ok 76 ▎ | |
| 613 :ok 41 ▏ | |
| 614 :ok 32 ▏ | |
| 615 :ok 54 ▏ | |
| 616 :ok 22 ▏ | |
| 617 :ok 29 ▏ | |
| 618 :ok 27 ▏ | |
| 619 :ok 55 ▏ | |
| 620 :ok 15 ▏ | |
| 621 :ok 16 ▏ | |
| 622 :ok 19 ▏ | |
| 623 :ok 26 ▏ | |
| 624 :ok 12 ▏ | |
| 625 :ok 15 ▏ | |
| 626 :ok 15 ▏ | |
| 627 :ok 3242 ██████▌ | |
| 628 :ok 13 ▏ | |
| 629 :ok 271 ▋ | |
| 630 :ok 24 ▏ | |
| 631 :ok 23 ▏ | |
| 632 :ok 24 ▏ | |
| 633 :ok 2745 █████▌ | |
| 634 :ok 21 ▏ | |
| 635 :ok 16 ▏ | |
| 636 :ok 18 ▏ | |
| 637 :ok 14 ▏ | |
| 638 :ok 11 ▏ | |
| 639 :ok 2255 ████▋ | |
| 640 :ok 25 ▏ | |
| 641 :ok 51 ▏ | |
| 642 :ok 53 ▏ | |
| 643 :ok 1742 ███▌ | |
| 644 :ok 23 ▏ | |
| 645 :ok 16 ▏ | |
| 646 :ok 1253 ██▋ | |
| 647 :ok 24 ▏ | |
| 648 :ok 23 ▏ | |
| 649 :ok 26 ▏ | |
| 650 :ok 24 ▏ | |
| 651 :ok 22 ▏ | |
| 652 :ok 23 ▏ | |
| 653 :ok 24 ▏ | |
| 654 :ok 24 ▏ | |
| 655 :ok 17 ▏ | |
| 656 :ok 13 ▏ | |
| 657 :ok 17 ▏ | |
| 658 :ok 16 ▏ | |
| 659 :ok 15 ▏ | |
| 660 :ok 17 ▏ | |
| 661 :ok 22 ▏ | |
| 662 :ok 22 ▏ | |
| 663 :ok 20 ▏ | |
| 664 :ok 19 ▏ | |
| 665 :ok 44 ▏ | |
| 666 :ok 41 ▏ | |
| 667 :ok 41 ▏ | |
| 668 :ok 15 ▏ | |
| 669 :ok 44 ▏ | |
| 670 :ok 18 ▏ | |
| 671 :ok 20 ▏ | |
| 672 :ok 16 ▏ | |
| 673 :ok 14 ▏ | |
| 674 :ok 15 ▏ | |
| 675 :ok 23 ▏ | |
| 676 :ok 19 ▏ | |
| 677 :ok 20 ▏ | |
| 678 :ok 21 ▏ | |
| 679 :ok 24 ▏ | |
| 680 :ok 21 ▏ | |
| 681 :ok 24 ▏ | |
| 682 :ok 25 ▏ | |
| 683 :ok 24 ▏ | |
| 684 :ok 23 ▏ | |
| 685 :ok 22 ▏ | |
| 686 :ok 21 ▏ | |
| 687 :ok 18 ▏ | |
| 688 :ok 15 ▏ | |
| 689 :ok 18 ▏ | |
| 690 :ok 20 ▏ | |
| 691 :ok 19 ▏ | |
| 692 :ok 23 ▏ | |
| 693 :ok 22 ▏ | |
| 694 :ok 22 ▏ | |
| 695 :ok 24 ▏ | |
| 696 :ok 24 ▏ | |
| 697 :ok 26 ▏ | |
| 698 :ok 27 ▏ | |
| 699 :ok 26 ▏ | |
| 700 :ok 20 ▏ | |
| 701 :ok 20 ▏ | |
| 702 :ok 21 ▏ | |
| 703 :ok 17 ▏ | |
| 704 :ok 19 ▏ | |
| 705 :ok 21 ▏ | |
| 706 :ok 26 ▏ | |
| 707 :ok 23 ▏ | |
| 708 :ok 23 ▏ | |
| 709 :ok 17 ▏ | |
| 710 :ok 21 ▏ | |
| 711 :ok 22 ▏ | |
| 712 :ok 26 ▏ | |
| 713 :ok 26 ▏ | |
| 714 :ok 28 ▏ | |
| 715 :ok 16 ▏ | |
| 716 :ok 20 ▏ | |
| 717 :ok 22 ▏ | |
| 718 :ok 18 ▏ | |
| 719 :ok 20 ▏ | |
| 720 :ok 23 ▏ | |
| 721 :ok 26 ▏ | |
| 722 :ok 24 ▏ | |
| 723 :ok 28 ▏ | |
| 724 :ok 27 ▏ | |
| 725 :ok 25 ▏ | |
| 726 :ok 23 ▏ | |
| 727 :ok 24 ▏ | |
| 728 :ok 19 ▏ | |
| 729 :ok 28 ▏ | |
| 730 :ok 18 ▏ | |
| 731 :ok 16 ▏ | |
| 732 :ok 21 ▏ | |
| 733 :ok 16 ▏ | |
| 734 :ok 15 ▏ | |
| 735 :ok 22 ▏ | |
| 736 :ok 23 ▏ | |
| 737 :ok 23 ▏ | |
| 738 :ok 19 ▏ | |
| 739 :ok 20 ▏ | |
| 740 :ok 19 ▏ | |
| 741 :ok 24 ▏ | |
| 742 :ok 22 ▏ | |
| 743 :ok 21 ▏ | |
| 744 :ok 24 ▏ | |
| 745 :ok 19 ▏ | |
| 746 :ok 18 ▏ | |
| 747 :ok 16 ▏ | |
| 748 :ok 19 ▏ | |
| 749 :ok 15 ▏ | |
| 750 :ok 21 ▏ | |
| 751 :ok 19 ▏ | |
| 752 :ok 21 ▏ | |
| 753 :ok 17 ▏ | |
| 754 :ok 21 ▏ | |
| 755 :ok 21 ▏ | |
| 756 :ok 25 ▏ | |
| 757 :ok 23 ▏ | |
| 758 :ok 23 ▏ | |
| 759 :ok 19 ▏ | |
| 760 :ok 20 ▏ | |
| 761 :ok 20 ▏ | |
| 762 :ok 15 ▏ | |
| 763 :ok 21 ▏ | |
| 764 :ok 21 ▏ | |
| 765 :ok 24 ▏ | |
| 766 :ok 27 ▏ | |
| 767 :ok 26 ▏ | |
| 768 :ok 21 ▏ | |
| 769 :ok 28 ▏ | |
| 770 :ok 23 ▏ | |
| 771 :ok 24 ▏ | |
| 772 :ok 26 ▏ | |
| 773 :ok 24 ▏ | |
| 774 :ok 23 ▏ | |
| 775 :ok 28 ▏ | |
| 776 :ok 29 ▏ | |
| 777 :ok 30 ▏ | |
| 778 :ok 24 ▏ | |
| 779 :ok 24 ▏ | |
| 780 :ok 22 ▏ | |
| 781 :ok 22 ▏ | |
| 782 :ok 24 ▏ | |
| 783 :ok 22 ▏ | |
| 784 :ok 18 ▏ | |
| 785 :ok 23 ▏ | |
| 786 :ok 24 ▏ | |
| 787 :ok 23 ▏ | |
| 788 :ok 23 ▏ | |
| 789 :ok 26 ▏ | |
| 790 :ok 17 ▏ | |
| 791 :ok 21 ▏ | |
| 792 :ok 60 ▏ | |
| 793 :ok 14 ▏ | |
| 794 :ok 21 ▏ | |
| 795 :ok 19 ▏ | |
| 796 :ok 17 ▏ | |
| 797 :ok 19 ▏ | |
| 798 :ok 21 ▏ | |
| 799 :ok 20 ▏ | |
| 800 :ok 25 ▏ | |
| 801 :ok 26 ▏ | |
| 802 :ok 27 ▏ | |
| 803 :ok 23 ▏ | |
| 804 :ok 23 ▏ | |
| 805 :ok 20 ▏ | |
| 806 :ok 19 ▏ | |
| 807 :ok 17 ▏ | |
| 808 :ok 18 ▏ | |
| 809 :ok 17 ▏ | |
| 810 :ok 20 ▏ | |
| 811 :ok 24 ▏ | |
| 812 :ok 18 ▏ | |
| 813 :ok 19 ▏ | |
| 814 :ok 23 ▏ | |
| 815 :ok 23 ▏ | |
| 816 :ok 24 ▏ | |
| 817 :ok 27 ▏ | |
| 818 :ok 20 ▏ | |
| 819 :ok 22 ▏ | |
| 820 :ok 18 ▏ | |
| 821 :ok 21 ▏ | |
| 822 :ok 21 ▏ | |
| 823 :ok 21 ▏ | |
| 824 :ok 15 ▏ | |
| 825 :ok 20 ▏ | |
| 826 :ok 21 ▏ | |
| 827 :ok 21 ▏ | |
| 828 :ok 22 ▏ | |
| 829 :ok 24 ▏ | |
| 830 :ok 25 ▏ | |
| 831 :ok 25 ▏ | |
| 832 :ok 26 ▏ | |
| 833 :ok 19 ▏ | |
| 834 :ok 21 ▏ | |
| 835 :ok 20 ▏ | |
| 836 :ok 21 ▏ | |
| 837 :ok 21 ▏ | |
| 838 :ok 25 ▏ | |
| 839 :ok 24 ▏ | |
| 840 :ok 63 ▎ | |
| 841 :ok 64 ▎ | |
| 842 :ok 65 ▎ | |
| 843 :ok 18 ▏ | |
| 844 :ok 21 ▏ | |
| 845 :ok 24 ▏ | |
| 846 :ok 25 ▏ | |
| 847 :ok 24 ▏ | |
| 848 :ok 26 ▏ | |
| 849 :ok 27 ▏ | |
| 850 :ok 16 ▏ | |
| 851 :ok 19 ▏ | |
| 852 :ok 17 ▏ | |
| 853 :ok 20 ▏ | |
| 854 :ok 19 ▏ | |
| 855 :ok 24 ▏ | |
| 856 :ok 24 ▏ | |
| 857 :ok 24 ▏ | |
| 858 :ok 23 ▏ | |
| 859 :ok 21 ▏ | |
| 860 :ok 25 ▏ | |
| 861 :ok 23 ▏ | |
| 862 :ok 27 ▏ | |
| 863 :ok 26 ▏ | |
| 864 :ok 25 ▏ | |
| 865 :ok 18 ▏ | |
| 866 :ok 20 ▏ | |
| 867 :ok 20 ▏ | |
| 868 :ok 16 ▏ | |
| 869 :ok 16 ▏ | |
| 870 :ok 19 ▏ | |
| 871 :ok 23 ▏ | |
| 872 :ok 24 ▏ | |
| 873 :ok 22 ▏ | |
| 874 :ok 20 ▏ | |
| 875 :ok 21 ▏ | |
| 876 :ok 27 ▏ | |
| 877 :ok 25 ▏ | |
| 878 :ok 21 ▏ | |
| 879 :ok 24 ▏ | |
| 880 :ok 22 ▏ | |
| 881 :ok 20 ▏ | |
| 882 :ok 17 ▏ | |
| 883 :ok 18 ▏ | |
| 884 :ok 18 ▏ | |
| 885 :ok 22 ▏ | |
| 886 :ok 22 ▏ | |
| 887 :ok 21 ▏ | |
| 888 :ok 21 ▏ | |
| 889 :ok 21 ▏ | |
| 890 :ok 25 ▏ | |
| 891 :ok 26 ▏ | |
| 892 :ok 25 ▏ | |
| 893 :ok 24 ▏ | |
| 894 :ok 24 ▏ | |
| 895 :ok 21 ▏ | |
| 896 :ok 26 ▏ | |
| 897 :ok 20 ▏ | |
| 898 :ok 23 ▏ | |
| 899 :ok 18 ▏ | |
| 900 :ok 22 ▏ | |
| 901 :ok 19 ▏ | |
| 902 :ok 19 ▏ | |
| 903 :ok 15 ▏ | |
| 904 :ok 18 ▏ | |
| 905 :ok 22 ▏ | |
| 906 :ok 24 ▏ | |
| 907 :ok 25 ▏ | |
| 908 :ok 27 ▏ | |
| 909 :ok 24 ▏ | |
| 910 :ok 16 ▏ | |
| 911 :ok 19 ▏ | |
| 912 :ok 17 ▏ | |
| 913 :ok 19 ▏ | |
| 914 :ok 20 ▏ | |
| 915 :ok 20 ▏ | |
| 916 :ok 22 ▏ | |
| 917 :ok 22 ▏ | |
| 918 :ok 16 ▏ | |
| 919 :ok 19 ▏ | |
| 920 :ok 27 ▏ | |
| 921 :ok 26 ▏ | |
| 922 :ok 26 ▏ | |
| 923 :ok 26 ▏ | |
| 924 :ok 22 ▏ | |
| 925 :ok 17 ▏ | |
| 926 :ok 18 ▏ | |
| 927 :ok 18 ▏ | |
| 928 :ok 18 ▏ | |
| 929 :ok 17 ▏ | |
| 930 :ok 21 ▏ | |
| 931 :ok 19 ▏ | |
| 932 :ok 23 ▏ | |
| 933 :ok 19 ▏ | |
| 934 :ok 22 ▏ | |
| 935 :ok 24 ▏ | |
| 936 :ok 21 ▏ | |
| 937 :ok 25 ▏ | |
| 938 :ok 19 ▏ | |
| 939 :ok 20 ▏ | |
| 940 :ok 19 ▏ | |
| 941 :ok 15 ▏ | |
| 942 :ok 19 ▏ | |
| 943 :ok 19 ▏ | |
| 944 :ok 19 ▏ | |
| 945 :ok 20 ▏ | |
| 946 :ok 24 ▏ | |
| 947 :ok 24 ▏ | |
| 948 :ok 21 ▏ | |
| 949 :ok 22 ▏ | |
| 950 :ok 20 ▏ | |
| 951 :ok 23 ▏ | |
| 952 :ok 26 ▏ | |
| 953 :ok 26 ▏ | |
| 954 :ok 18 ▏ | |
| 955 :ok 18 ▏ | |
| 956 :ok 16 ▏ | |
| 957 :ok 21 ▏ | |
| 958 :ok 16 ▏ | |
| 959 :ok 13 ▏ | |
| 960 :ok 21 ▏ | |
| 961 :ok 19 ▏ | |
| 962 :ok 22 ▏ | |
| 963 :ok 19 ▏ | |
| 964 :ok 23 ▏ | |
| 965 :ok 24 ▏ | |
| 966 :ok 28 ▏ | |
| 967 :ok 24 ▏ | |
| 968 :ok 24 ▏ | |
| 969 :ok 25 ▏ | |
| 970 :ok 15 ▏ | |
| 971 :ok 10 ▏ | |
| 972 :ok 12 ▏ | |
| 973 :ok 18 ▏ | |
| 974 :ok 13 ▏ | |
| 975 :ok 20 ▏ | |
| 976 :ok 21 ▏ | |
| 977 :ok 20 ▏ | |
| 978 :ok 20 ▏ | |
| 979 :ok 20 ▏ | |
| 980 :ok 18 ▏ | |
| 981 :ok 18 ▏ | |
| 982 :ok 20 ▏ | |
| 983 :ok 15 ▏ | |
| 984 :ok 19 ▏ | |
| 985 :ok 19 ▏ | |
| 986 :ok 19 ▏ | |
| 987 :ok 21 ▏ | |
| 988 :ok 20 ▏ | |
| 989 :ok 19 ▏ | |
| 990 :ok 21 ▏ | |
| 991 :ok 25 ▏ | |
| 992 :ok 23 ▏ | |
| 993 :ok 19 ▏ | |
| 994 :ok 23 ▏ | |
| 995 :ok 22 ▏ | |
| 996 :ok 23 ▏ | |
| 997 :ok 24 ▏ | |
| 998 :ok 27 ▏ | |
| 999 :ok 26 ▏ | |
| 0 unrecoverable timeouts | |
| Collecting results. | |
| Here's the set of writes for which a != -b; e.g. row isolation did not hold: | |
| ({:id 114, :a 1570376346, :b -812690858} {:id 55, :a 1706671853, :b -809618844} {:id 324, :a 1570376346, :b -809618844} {:id 380, :a 1706671853, :b -809618844} {:id 857, :a 1570376346, :b -436112969} {:id 914, :a 1570376346, :b -812690858} {:id 780, :a 1706671853, :b -809618844} {:id 185, :a 812690858, :b -809618844} {:id 369, :a 812690858, :b -809618844} {:id 959, :a 812690858, :b -809618844} {:id 323, :a 1570376346, :b -812690858} {:id 514, :a 812690858, :b -809618844} {:id 467, :a 1570376346, :b -436112969} {:id 398, :a 1570376346, :b -812690858} {:id 414, :a 1570376346, :b -809618844} {:id 500, :a 1706671853, :b -809618844} {:id 570, :a 1706671853, :b -809618844} {:id 883, :a 1570376346, :b -812690858} {:id 835, :a 1706671853, :b -809618844} {:id 864, :a 812690858, :b -809618844} {:id 113, :a 1570376346, :b -812690858} {:id 390, :a 1706671853, :b -812690858} {:id 280, :a 1706671853, :b -809618844} {:id 344, :a 812690858, :b -809618844} {:id 257, :a 1570376346, :b -436112969} {:id 0, :a 1706671853, :b -809618844} {:id 512, :a 1570376346, :b -436112969} {:id 440, :a 1706671853, :b -809618844} {:id 697, :a 1570376346, :b -436112969} {:id 667, :a 1570376346, :b -436112969} {:id 84, :a 812690858, :b -809618844} {:id 403, :a 1570376346, :b -812690858} {:id 74, :a 812690858, :b -809618844} {:id 345, :a 1706671853, :b -809618844} {:id 372, :a 1570376346, :b -436112969} {:id 492, :a 1570376346, :b -436112969} {:id 833, :a 1570376346, :b -812690858} {:id 555, :a 1706671853, :b -812690858} {:id 775, :a 1706671853, :b -812690858} {:id 827, :a 1570376346, :b -436112969} {:id 809, :a 1570376346, :b -809618844} {:id 575, :a 1706671853, :b -809618844} {:id 742, :a 1570376346, :b -436112969} {:id 584, :a 812690858, :b -809618844} {:id 198, :a 1570376346, :b -812690858} {:id 545, :a 1706671853, :b -809618844} {:id 996, :a 1706671853, :b -809618844} {:id 494, :a 812690858, :b -436112969} {:id 442, :a 1570376346, :b -436112969} {:id 524, :a 812690858, :b -809618844} {:id 405, :a 1706671853, :b -809618844} {:id 977, :a 1570376346, :b -436112969} {:id 258, :a 1570376346, :b -812690858} {:id 349, :a 812690858, :b -809618844} {:id 683, :a 812690858, :b -436112969} {:id 180, :a 1706671853, :b -809618844} {:id 127, :a 1570376346, :b -436112969} {:id 699, :a 812690858, :b -809618844} {:id 629, :a 812690858, :b -809618844} {:id 604, :a 812690858, :b -809618844} {:id 175, :a 1706671853, :b -812690858} {:id 589, :a 812690858, :b -809618844} {:id 995, :a 1706671853, :b -809618844} {:id 329, :a 812690858, :b -809618844} {:id 470, :a 1706671853, :b -809618844} {:id 318, :a 1570376346, :b -812690858} {:id 784, :a 812690858, :b -809618844} {:id 789, :a 1570376346, :b -809618844} {:id 805, :a 1706671853, :b -809618844} {:id 680, :a 1706671853, :b -809618844} {:id 734, :a 1570376346, :b -809618844} {:id 989, :a 1570376346, :b -812690858} {:id 422, :a 1570376346, :b -436112969} {:id 275, :a 1706671853, :b -809618844}) | |
| Writes completed in 100.04 seconds | |
| 1000 total | |
| 976 acknowledged | |
| 926 survivors | |
| 74 acknowledged writes lost! (╯°□°）╯︵ ┻━┻ | |
| 0 55 74 84 113 114 ... 914 959 977 989 995 996 | |
| 24 unacknowledged writes found! ヽ(´ー｀)ノ | |
| 118 121 142 145 146 147 ... 190 220 235 245 246 250 | |
| 0.976 ack rate | |
| 0.07581967 loss rate | |
| 0.024590164 unacknowledged but successful rate |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| (defn isolation-app | |
| "This app tests whether or not it is possible to consistently update multiple | |
| cells in a row, such that either *both* writes are visible together, or | |
| *neither* is. | |
| Each client picks a random int identifier to distinguish itself from the | |
| other clients. It tries to write this identifier to cell A, and -identifier | |
| to cell B. The write is considered successful if A=-B. It is unsuccessful if | |
| A is *not* equal to -B; e.g. our updates were not isolated. | |
| 'concurrency defines the number of writes made to each row. At concurrency 5, | |
| element 10 results in writes to rows 10, 9, 8, 7, and 6." | |
| [opts] | |
| (let [table "isolation_app" | |
| concurrency 5 | |
| client-id (rand-int Integer/MAX_VALUE) | |
| cluster (client/build-cluster {:contact-points [(:host opts)] | |
| :port 9042}) | |
| session (client/connect cluster)] | |
| (reify SetApp | |
| (setup [app] | |
| (teardown app) | |
| (ensure-keyspace session) | |
| (cql/create-table session table | |
| (column-definitions {:id :int | |
| :a :int | |
| :b :int | |
| :primary-key [:id]}))) | |
| (add [app element] | |
| (client/with-consistency-level ConsistencyLevel/ANY | |
| (dotimes [i concurrency] | |
| (let [e (- element i)] | |
| (when (<= 0 e) | |
| (client/execute | |
| session | |
| (->> (insert-query table | |
| {:id e | |
| :a client-id | |
| :b (- client-id)}) | |
| ; If you force timestamp collisions instead of letting | |
| ; them happen naturally, you can reliably cause | |
| ; conflicts in 99% of rows! :D | |
| ; (using :timestamp 1) | |
| queries | |
| batch-query | |
| client/render-query) | |
| :prepared qbits.hayt.cql/*prepared-statement*))))) | |
| ok) | |
| (results [app] | |
| (client/with-consistency-level ConsistencyLevel/ALL | |
| (->> (cql/select session table) | |
| (remove #(= (:a %) (- (:b %)))) | |
| prn | |
| dorun) | |
| (->> (cql/select session table) | |
| (filter #(= (:a %) (- (:b %)))) | |
| (map :id)))) | |
| (teardown [app] | |
| (drop-keyspace session))))) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment