Skip to content

Instantly share code, notes, and snippets.

@jasobrown
Created July 24, 2018 14:39
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save jasobrown/e7ecf6d0bf875d1f4a08ee06ac7eaba0 to your computer and use it in GitHub Desktop.
Save jasobrown/e7ecf6d0bf875d1f4a08ee06ac7eaba0 to your computer and use it in GitHub Desktop.
dump of @jasobrown's dtest fail analysis as os late July 2018
CASSANDRA 3.11 DTEST FAILS ::: https://circleci.com/workflow-run/c19fc633-7dd3-427b-b914-257409272274
no-vnodes:: 5 fails
1) internal - pytest
TypeError: unhashable type: 'InvalidRequestException'
2) test_describecluster_more_information_three_datacenters - nodetool_test.TestNodetool
> assert 'Live: 6' in out_node1_dc1
E AssertionError: assert 'Live: 6' in 'Cluster Information:\n\tName: test\n\tSnitch: org.apache.cassandra.locator.PropertyFileSnitch\n\tDynamicEndPointSnitc...ons:\n\t\t185761b8-eaf9-3380-8e13-45e608976901: [127.0.0.1, 127.0.0.2, 127.0.0.3, 127.0.0.4, 127.0.0.5, 127.0.0.6]\n\n'
3) test_failure_threshold_deletions - paging_test.TestPagingWithDeletions
E cassandra.ReadFailure: Error from server: code=1300 [Replica(s) failed to execute read] message="Operation failed - received 0 responses and 1 failures" info={'consistency': 'ALL', 'required_responses': 2, 'received_responses': 0, 'failures': 1}
4) test_closing_connections - thrift_hsha_test.TestThriftHSHA
ccmlib.node.ToolError: Subprocess ['nodetool', '-h', 'localhost', '-p', '7100', 'enablethrift'] exited with non-zero status; exit status: 2; stderr: error: Could not create ServerSocket on address /127.0.0.1:9160. -- StackTrace -- org.apache.thrift.transport.TTransportException: Could not create ServerSocket on address /127.0.0.1:9160. at org.apache.thrift.transport.TNonblockingServerSocket.<init>(TNonblockingServerSocket.java:96) at org.apache.thrift.transport.TNonblockingServerSocket.<init>(TNonblockingServerSocket.java:79) at org.apache.thrift.transport.TNonblockingServerSocket.<init>(TNonblockingServerSocket.java:75) at org.apache.cassandra.thrift.TCustomNonblockingServerSocket.<init>(TCustomNonblockingServerSocket.java:39) at org.apache.cassandra.thrift.THsHaDisruptorServer$Factory.buildTServer(THsHaDisruptorServer.java:80) at org.apache.cassandra.thrift.TServerCustomFactory.buildTServer(TServerCustomFactory.java:55) at org.apache.cassandra.thrift.ThriftServer$ThriftServerThread.<init>(ThriftServer.java:128) at org.apache.cassandra.thrift.ThriftServer.start(ThriftServer.java:55) at org.apache.cassandra.service.StorageService.startRPCServer(StorageService.java:364) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:71) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:275) at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112) at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46) at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237) at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138) at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819) at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801) at javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1468) at javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:76) at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1309) at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1401) at javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:829) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:357) at sun.rmi.transport.Transport$1.run(Transport.java:200) at sun.rmi.transport.Transport$1.run(Transport.java:197) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.Transport.serviceCall(Transport.java:196) at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:568) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:683) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
5) test_mutation_v5 - write_failures_test.TestWriteFailures
s = b'\x00\x00'
> unpack = lambda s: packer.unpack(s)[0]
E struct.error: unpack requires a buffer of 4 bytes
--------------------------------
vnodes:: 6 failures
1) internal - pytest
TypeError: unhashable type: 'InvalidRequestException'
2) test_system_auth_ks_is_alterable - auth_test.TestAuth
test teardown failure
Unexpected error found in node logs (see stdout for full details). Errors: [ERROR [Native-Transport-Requests-1] 2018-07-23 18:14:34,585 Message.java:629 - Unexpected exception during request; channel = [id: 0x0ffc99f5, L:/127.0.0.3:9042 - R:/127.0.0.1:54516]
java.lang.RuntimeException: org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
at org.apache.cassandra.auth.CassandraRoleManager.getRole(CassandraRoleManager.java:518) ~[main/:na]
at org.apache.cassandra.auth.CassandraRoleManager.canLogin(CassandraRoleManager.java:312) ~[main/:na]
at org.apache.cassandra.service.ClientState.login(ClientState.java:281) ~[main/:na]
at org.apache.cassandra.transport.messages.AuthResponse.execute(AuthResponse.java:80) ~[main/:na]
at org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:517) [main/:na]
at org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:410) [main/:na]
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105) [netty-all-4.0.44.Final.jar:4.0.44.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:357) [netty-all-4.0.44.Final.jar:4.0.44.Final]
at io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:35) [netty-all-4.0.44.Final.jar:4.0.44.Final]
at io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:348) [netty-all-4.0.44.Final.jar:4.0.44.Final]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_154-cassandra]
at org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:162) [main/:na]
at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:109) [main/:na]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_154-cassandra]
Caused by: org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
at org.apache.cassandra.db.ConsistencyLevel.assureSufficientLiveNodes(ConsistencyLevel.java:334) ~[main/:na]
at org.apache.cassandra.service.AbstractReadExecutor.getReadExecutor(AbstractReadExecutor.java:162) ~[main/:na]
at org.apache.cassandra.service.StorageProxy$SinglePartitionReadLifecycle.<init>(StorageProxy.java:1779) ~[main/:na]
at org.apache.cassandra.service.StorageProxy.fetchRows(StorageProxy.java:1741) ~[main/:na]
at org.apache.cassandra.service.StorageProxy.readRegular(StorageProxy.java:1684) ~[main/:na]
at org.apache.cassandra.service.StorageProxy.read(StorageProxy.java:1599) ~[main/:na]
at org.apache.cassandra.db.SinglePartitionReadCommand$Group.execute(SinglePartitionReadCommand.java:1176) ~[main/:na]
at org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:315) ~[main/:na]
at org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:285) ~[main/:na]
at org.apache.cassandra.auth.CassandraRoleManager.getRoleFromTable(CassandraRoleManager.java:526) ~[main/:na]
at org.apache.cassandra.auth.CassandraRoleManager.getRole(CassandraRoleManager.java:508) ~[main/:na]
... 13 common frames omitted, ERROR [Native-Transport-Requests-3] 2018-07-23 18:14:35,759 Message.java:629 - Unexpected exception during request; channel = [id: 0x3bf15467, L:/127.0.0.3:9042 - R:/127.0.0.1:54528]
3) test_describecluster_more_information_three_datacenters - nodetool_test.TestNodetool
> assert 'Live: 6' in out_node1_dc1
E AssertionError: assert 'Live: 6' in 'Cluster Information:\n\tName: test\n\tSnitch: org.apache.cassandra.locator.PropertyFileSnitch\n\tDynamicEndPointSnitc...ons:\n\t\t256a063d-12f8-32ae-b4ed-b486d82927b2: [127.0.0.1, 127.0.0.2, 127.0.0.3, 127.0.0.4, 127.0.0.5, 127.0.0.6]\n\n'
4) test_failure_threshold_deletions - paging_test.TestPagingWithDeletions
> ???
E cassandra.ReadFailure: Error from server: code=1300 [Replica(s) failed to execute read] message="Operation failed - received 0 responses and 2 failures" info={'consistency': 'ALL', 'required_responses': 2, 'received_responses': 0, 'failures': 2}
5) test_closing_connections - thrift_hsha_test.TestThriftHSHA
ccmlib.node.ToolError: Subprocess ['nodetool', '-h', 'localhost', '-p', '7100', 'enablethrift'] exited with non-zero status; exit status: 2; stderr: error: Could not create ServerSocket on address /127.0.0.1:9160. -- StackTrace -- org.apache.thrift.transport.TTransportException: Could not create ServerSocket on address /127.0.0.1:9160. at org.apache.thrift.transport.TNonblockingServerSocket.<init>(TNonblockingServerSocket.java:96) at org.apache.thrift.transport.TNonblockingServerSocket.<init>(TNonblockingServerSocket.java:79) at org.apache.thrift.transport.TNonblockingServerSocket.<init>(TNonblockingServerSocket.java:75) at org.apache.cassandra.thrift.TCustomNonblockingServerSocket.<init>(TCustomNonblockingServerSocket.java:39) at org.apache.cassandra.thrift.THsHaDisruptorServer$Factory.buildTServer(THsHaDisruptorServer.java:80) at org.apache.cassandra.thrift.TServerCustomFactory.buildTServer(TServerCustomFactory.java:55) at org.apache.cassandra.thrift.ThriftServer$ThriftServerThread.<init>(ThriftServer.java:128) at org.apache.cassandra.thrift.ThriftServer.start(ThriftServer.java:55) at org.apache.cassandra.service.StorageService.startRPCServer(StorageService.java:364) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:71) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:275) at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112) at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46) at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237) at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138) at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819) at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801) at javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1468) at javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:76) at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1309) at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1401) at javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:829) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:357) at sun.rmi.transport.Transport$1.run(Transport.java:200) at sun.rmi.transport.Transport$1.run(Transport.java:197) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.Transport.serviceCall(Transport.java:196) at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:568) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:683) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
6) test_mutation_v5 - write_failures_test.TestWriteFailures
s = b'\x00\x00'
> unpack = lambda s: packer.unpack(s)[0]
E struct.error: unpack requires a buffer of 4 bytes
**********************************************
**********************************************
**********************************************
CASSANDRA 3.0 DTEST FAILS ::: https://circleci.com/workflow-run/4e2cf1e9-7064-48ce-9912-2709ae839b8a
UTEST fail:
testObsoletionPerformance - org.apache.cassandra.db.lifecycle.HelpersTest
junit.framework.AssertionFailedError:
at org.apache.cassandra.db.lifecycle.HelpersTest.testObsoletionPerformance(HelpersTest.java:202)
no-vnodes:: 16 failures
1) internal - pytest
TypeError: unhashable type: 'InvalidRequestException'
2) test_permissions_caching - auth_test.TestAuth
if remaining <= 0:
> raise TimeoutExpired(self.args, timeout)
E subprocess.TimeoutExpired: Command 'sudo /bin/sync' timed out after 600 seconds
3) test_system_auth_ks_is_alterable - auth_test.TestAuth
test teardown failure
Unexpected error found in node logs (see stdout for full details). Errors: [ERROR [SharedPool-Worker-3] 2018-07-23 18:52:05,981 Message.java:625 - Unexpected exception during request; channel = [id: 0x2205ed0d, L:/127.0.0.3:9042 - R:/127.0.0.1:53700]
java.lang.RuntimeException: org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
at org.apache.cassandra.auth.CassandraRoleManager.getRole(CassandraRoleManager.java:517) ~[main/:na]
at org.apache.cassandra.auth.CassandraRoleManager.canLogin(CassandraRoleManager.java:311) ~[main/:na]
at org.apache.cassandra.service.ClientState.login(ClientState.java:280) ~[main/:na]
at org.apache.cassandra.transport.messages.AuthResponse.execute(AuthResponse.java:79) ~[main/:na]
at org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:513) [main/:na]
at org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:407) [main/:na]
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105) [netty-all-4.0.44.Final.jar:4.0.44.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:357) [netty-all-4.0.44.Final.jar:4.0.44.Final]
at io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:35) [netty-all-4.0.44.Final.jar:4.0.44.Final]
at io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:348) [netty-all-4.0.44.Final.jar:4.0.44.Final]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_154-cassandra]
at org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164) [main/:na]
at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) [main/:na]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_154-cassandra]
Caused by: org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
at org.apache.cassandra.db.ConsistencyLevel.assureSufficientLiveNodes(ConsistencyLevel.java:334) ~[main/:na]
at org.apache.cassandra.service.AbstractReadExecutor.getReadExecutor(AbstractReadExecutor.java:162) ~[main/:na]
at org.apache.cassandra.service.StorageProxy$SinglePartitionReadLifecycle.<init>(StorageProxy.java:1693) ~[main/:na]
at org.apache.cassandra.service.StorageProxy.fetchRows(StorageProxy.java:1656) ~[main/:na]
at org.apache.cassandra.service.StorageProxy.readRegular(StorageProxy.java:1603) ~[main/:na]
at org.apache.cassandra.service.StorageProxy.read(StorageProxy.java:1522) ~[main/:na]
at org.apache.cassandra.db.SinglePartitionReadCommand$Group.execute(SinglePartitionReadCommand.java:1133) ~[main/:na]
at org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:303) ~[main/:na]
at org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:264) ~[main/:na]
at org.apache.cassandra.auth.CassandraRoleManager.getRoleFromTable(CassandraRoleManager.java:525) ~[main/:na]
at org.apache.cassandra.auth.CassandraRoleManager.getRole(CassandraRoleManager.java:507) ~[main/:na]
... 13 common frames omitted, ERROR [SharedPool-Worker-3] 2018-07-23 18:52:05,981 Message.java:625 - Unexpected exception during request; channel = [id: 0x2205ed0d, L:/127.0.0.3:9042 - R:/127.0.0.1:53700]
4) test_simple_bootstrap_nodata - bootstrap_test.TestBootstrap
test setup failure
if remaining <= 0:
> raise TimeoutExpired(self.args, timeout)
E subprocess.TimeoutExpired: Command 'sudo /bin/sync' timed out after 600 seconds
/usr/lib/python3.6/subprocess.py:1449: TimeoutExpired
5) test_functional - global_row_key_cache_test.TestGlobalRowKeyCache
test teardown failure
Unexpected error found in node logs (see stdout for full details). Errors: [WARN [main] 2018-07-23 18:53:10,075 Uns.java:169 - Failed to load Java8 implementation ohc-core-j8 : java.lang.NoSuchMethodException: org.caffinitas.ohc.linked.UnsExt8.<init>(java.lang.Class), WARN [main] 2018-07-23 18:53:56,966 Uns.java:169 - Failed to load Java8 implementation ohc-core-j8 : java.lang.NoSuchMethodException: org.caffinitas.ohc.linked.UnsExt8.<init>(java.lang.Class), WARN [main] 2018-07-23 18:55:54,508 Uns.java:169 - Failed to load Java8 implementation ohc-core-j8 : java.lang.NoSuchMethodException: org.caffinitas.ohc.linked.UnsExt8.<init>(java.lang.Class), WARN [main] 2018-07-23 18:56:42,688 Uns.java:169 - Failed to load Java8 implementation ohc-core-j8 : java.lang.NoSuchMethodException: org.caffinitas.ohc.linked.UnsExt8.<init>(java.lang.Class), WARN [main] 2018-07-23 18:53:10,075 Uns.java:169 - Failed to load Java8 implementation ohc-core-j8 : java.lang.NoSuchMethodException: org.caffinitas.ohc.linked.UnsExt8.<init>(java.lang.Class)]
6) test_describecluster_more_information_three_datacenters - nodetool_test.TestNodetool
> assert 'Live: 6' in out_node1_dc1
E AssertionError: assert 'Live: 6' in 'Cluster Information:\n\tName: test\n\tSnitch: org.apache.cassandra.locator.DynamicEndpointSnitch\n\tPartitioner: org....ons:\n\t\tb7f4adb5-55f6-39fe-b751-d4bd7e5f5719: [127.0.0.1, 127.0.0.2, 127.0.0.3, 127.0.0.4, 127.0.0.5, 127.0.0.6]\n\n'
nodetool_test.py:376: AssertionError
7) test_sstableofflinerelevel - offline_tools_test.TestOfflineTools
> assert re.search("L0=1", output)
E AssertionError: assert None
E + where None = <function search at 0x7f8f516e3e18>('L0=1', 'New leveling: \nL0=0\nL1 10\n')
E + where <function search at 0x7f8f516e3e18> = re.search
offline_tools_test.py:160: AssertionError
8) test_alter_rf_and_run_read_repair - read_repair_test.TestReadRepair
if res != expected:
> raise NotRepairedException()
E read_repair_test.NotRepairedException
read_repair_test.py:204: NotRepairedException
9) test_read_repair_chance - read_repair_test.TestReadRepair
if res != expected:
> raise NotRepairedException()
E read_repair_test.NotRepairedException
read_repair_test.py:204: NotRepairedException
10) test_archive_commitlog - snapshot_test.TestArchiveCommitlog
assert (
time.time() <= stop_time), "It's been over a {s}s and we haven't written a new " + \
> "commitlog segment. Something is wrong.".format(s=timeout)
E AssertionError: It's been over a {s}s and we haven't written a new commitlog segment. Something is wrong.
tools/hacks.py:61: AssertionError
11) test_archive_commitlog_point_in_time - snapshot_test.TestArchiveCommitlog
rate_limited_debug_logger(' commitlog-advancing load step has lasted {s:.2f}s'.format(s=elapsed))
assert (
time.time() <= stop_time), "It's been over a {s}s and we haven't written a new " + \
> "commitlog segment. Something is wrong.".format(s=timeout)
E AssertionError: It's been over a {s}s and we haven't written a new commitlog segment. Something is wrong.
tools/hacks.py:61: AssertionError
12) test_archive_commitlog_point_in_time_with_active_commitlog - snapshot_test.TestArchiveCommitlog
assert (
time.time() <= stop_time), "It's been over a {s}s and we haven't written a new " + \
> "commitlog segment. Something is wrong.".format(s=timeout)
E AssertionError: It's been over a {s}s and we haven't written a new commitlog segment. Something is wrong.
tools/hacks.py:61: AssertionError
13) test_archive_commitlog_point_in_time_with_active_commitlog_ln - snapshot_test.TestArchiveCommitlog
assert (
time.time() <= stop_time), "It's been over a {s}s and we haven't written a new " + \
> "commitlog segment. Something is wrong.".format(s=timeout)
E AssertionError: It's been over a {s}s and we haven't written a new commitlog segment. Something is wrong.
tools/hacks.py:61: AssertionError
14) test_archive_commitlog_with_active_commitlog - snapshot_test.TestArchiveCommitlog
assert (
time.time() <= stop_time), "It's been over a {s}s and we haven't written a new " + \
> "commitlog segment. Something is wrong.".format(s=timeout)
E AssertionError: It's been over a {s}s and we haven't written a new commitlog segment. Something is wrong.
tools/hacks.py:61: AssertionError
15) test_dont_archive_commitlog - snapshot_test.TestArchiveCommitlog
assert (
time.time() <= stop_time), "It's been over a {s}s and we haven't written a new " + \
> "commitlog segment. Something is wrong.".format(s=timeout)
E AssertionError: It's been over a {s}s and we haven't written a new commitlog segment. Something is wrong.
tools/hacks.py:61: AssertionError
16) test_closing_connections - thrift_hsha_test.TestThriftHSHA
ccmlib.node.ToolError: Subprocess ['nodetool', '-h', 'localhost', '-p', '7100', 'enablethrift'] exited with non-zero status; exit status: 2; stderr: error: Could not create ServerSocket on address /127.0.0.1:9160. -- StackTrace -- org.apache.thrift.transport.TTransportException: Could not create ServerSocket on address /127.0.0.1:9160. at org.apache.thrift.transport.TNonblockingServerSocket.<init>(TNonblockingServerSocket.java:96) at org.apache.thrift.transport.TNonblockingServerSocket.<init>(TNonblockingServerSocket.java:79) at org.apache.thrift.transport.TNonblockingServerSocket.<init>(TNonblockingServerSocket.java:75) at org.apache.cassandra.thrift.TCustomNonblockingServerSocket.<init>(TCustomNonblockingServerSocket.java:39) at org.apache.cassandra.thrift.THsHaDisruptorServer$Factory.buildTServer(THsHaDisruptorServer.java:80) at org.apache.cassandra.thrift.TServerCustomFactory.buildTServer(TServerCustomFactory.java:55) at org.apache.cassandra.thrift.ThriftServer$ThriftServerThread.<init>(ThriftServer.java:131) at org.apache.cassandra.thrift.ThriftServer.start(ThriftServer.java:58) at org.apache.cassandra.service.StorageService.startRPCServer(StorageService.java:355) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:71) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:275) at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112) at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46) at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237) at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138) at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819) at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801) at javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1468) at javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:76) at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1309) at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1401) at javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:829) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:357) at sun.rmi.transport.Transport$1.run(Transport.java:200) at sun.rmi.transport.Transport$1.run(Transport.java:197) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.Transport.serviceCall(Transport.java:196) at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:568) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:683) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
vnodes:: 14 failures
1) internal - pytest
TypeError: unhashable type: 'InvalidRequestException'
2) test_system_auth_ks_is_alterable - auth_test.TestAuth
test teardown failure
Unexpected error found in node logs (see stdout for full details). Errors: [ERROR [SharedPool-Worker-2] 2018-07-23 18:41:23,568 Message.java:625 - Unexpected exception during request; channel = [id: 0x23a7105f, L:/127.0.0.3:9042 - R:/127.0.0.1:36894]
java.lang.RuntimeException: org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
at org.apache.cassandra.auth.CassandraRoleManager.getRole(CassandraRoleManager.java:517) ~[main/:na]
at org.apache.cassandra.auth.CassandraRoleManager.canLogin(CassandraRoleManager.java:311) ~[main/:na]
at org.apache.cassandra.service.ClientState.login(ClientState.java:280) ~[main/:na]
at org.apache.cassandra.transport.messages.AuthResponse.execute(AuthResponse.java:79) ~[main/:na]
at org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:513) [main/:na]
at org.apache.cassandra.transport.Message$Dispatcher.channelRead0(Message.java:407) [main/:na]
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105) [netty-all-4.0.44.Final.jar:4.0.44.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:357) [netty-all-4.0.44.Final.jar:4.0.44.Final]
at io.netty.channel.AbstractChannelHandlerContext.access$600(AbstractChannelHandlerContext.java:35) [netty-all-4.0.44.Final.jar:4.0.44.Final]
at io.netty.channel.AbstractChannelHandlerContext$7.run(AbstractChannelHandlerContext.java:348) [netty-all-4.0.44.Final.jar:4.0.44.Final]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_154-cassandra]
at org.apache.cassandra.concurrent.AbstractLocalAwareExecutorService$FutureTask.run(AbstractLocalAwareExecutorService.java:164) [main/:na]
at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) [main/:na]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_154-cassandra]
Caused by: org.apache.cassandra.exceptions.UnavailableException: Cannot achieve consistency level QUORUM
at org.apache.cassandra.db.ConsistencyLevel.assureSufficientLiveNodes(ConsistencyLevel.java:334) ~[main/:na]
at org.apache.cassandra.service.AbstractReadExecutor.getReadExecutor(AbstractReadExecutor.java:162) ~[main/:na]
at org.apache.cassandra.service.StorageProxy$SinglePartitionReadLifecycle.<init>(StorageProxy.java:1693) ~[main/:na]
at org.apache.cassandra.service.StorageProxy.fetchRows(StorageProxy.java:1656) ~[main/:na]
at org.apache.cassandra.service.StorageProxy.readRegular(StorageProxy.java:1603) ~[main/:na]
at org.apache.cassandra.service.StorageProxy.read(StorageProxy.java:1522) ~[main/:na]
at org.apache.cassandra.db.SinglePartitionReadCommand$Group.execute(SinglePartitionReadCommand.java:1133) ~[main/:na]
at org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:303) ~[main/:na]
at org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:264) ~[main/:na]
at org.apache.cassandra.auth.CassandraRoleManager.getRoleFromTable(CassandraRoleManager.java:525) ~[main/:na]
at org.apache.cassandra.auth.CassandraRoleManager.getRole(CassandraRoleManager.java:507) ~[main/:na]
... 13 common frames omitted, ERROR [SharedPool-Worker-2] 2018-07-23 18:41:23,568 Message.java:625 - Unexpected exception during request; channel = [id: 0x23a7105f, L:/127.0.0.3:9042 - R:/127.0.0.1:36894]
3) test_functional - global_row_key_cache_test.TestGlobalRowKeyCache
test teardown failure
Unexpected error found in node logs (see stdout for full details). Errors: [WARN [main] 2018-07-23 18:42:26,918 Uns.java:169 - Failed to load Java8 implementation ohc-core-j8 : java.lang.NoSuchMethodException: org.caffinitas.ohc.linked.UnsExt8.<init>(java.lang.Class), WARN [main] 2018-07-23 18:43:15,689 Uns.java:169 - Failed to load Java8 implementation ohc-core-j8 : java.lang.NoSuchMethodException: org.caffinitas.ohc.linked.UnsExt8.<init>(java.lang.Class), WARN [main] 2018-07-23 18:45:15,361 Uns.java:169 - Failed to load Java8 implementation ohc-core-j8 : java.lang.NoSuchMethodException: org.caffinitas.ohc.linked.UnsExt8.<init>(java.lang.Class), WARN [main] 2018-07-23 18:46:05,702 Uns.java:169 - Failed to load Java8 implementation ohc-core-j8 : java.lang.NoSuchMethodException: org.caffinitas.ohc.linked.UnsExt8.<init>(java.lang.Class), WARN [main] 2018-07-23 18:42:26,918 Uns.java:169 - Failed to load Java8 implementation ohc-core-j8 : java.lang.NoSuchMethodException: org.caffinitas.ohc.linked.UnsExt8.<init>(java.lang.Class)]
4) test_describecluster_more_information_three_datacenters - nodetool_test.TestNodetool
> assert 'Live: 6' in out_node1_dc1
E AssertionError: assert 'Live: 6' in 'Cluster Information:\n\tName: test\n\tSnitch: org.apache.cassandra.locator.DynamicEndpointSnitch\n\tPartitioner: org....ons:\n\t\t74e27a5b-86fa-37b4-abd1-edb8a0e6948c: [127.0.0.1, 127.0.0.2, 127.0.0.3, 127.0.0.4, 127.0.0.5, 127.0.0.6]\n\n'
5) test_sstableofflinerelevel - offline_tools_test.TestOfflineTools
> assert re.search("L0=1", output)
E AssertionError: assert None
E + where None = <function search at 0x7fe2b03bce18>('L0=1', 'New leveling: \nL0=0\nL1 10\n')
E + where <function search at 0x7fe2b03bce18> = re.search
6) test_alter_rf_and_run_read_repair - read_repair_test.TestReadRepair
> raise NotRepairedException()
E read_repair_test.NotRepairedException
read_repair_test.py:204: NotRepairedException
7) test_read_repair_chance - read_repair_test.TestReadRepair
if res != expected:
> raise NotRepairedException()
E read_repair_test.NotRepairedException
read_repair_test.py:204: NotRepairedException
8) test_archive_commitlog - snapshot_test.TestArchiveCommitlog
assert (
time.time() <= stop_time), "It's been over a {s}s and we haven't written a new " + \
> "commitlog segment. Something is wrong.".format(s=timeout)
E AssertionError: It's been over a {s}s and we haven't written a new commitlog segment. Something is wrong.
tools/hacks.py:61: AssertionError
9) test_archive_commitlog_point_in_time - snapshot_test.TestArchiveCommitlog
assert (
time.time() <= stop_time), "It's been over a {s}s and we haven't written a new " + \
> "commitlog segment. Something is wrong.".format(s=timeout)
E AssertionError: It's been over a {s}s and we haven't written a new commitlog segment. Something is wrong.
tools/hacks.py:61: AssertionError
10) test_archive_commitlog_point_in_time_with_active_commitlog - snapshot_test.TestArchiveCommitlog
assert (
time.time() <= stop_time), "It's been over a {s}s and we haven't written a new " + \
> "commitlog segment. Something is wrong.".format(s=timeout)
E AssertionError: It's been over a {s}s and we haven't written a new commitlog segment. Something is wrong.
tools/hacks.py:61: AssertionError
11) test_archive_commitlog_point_in_time_with_active_commitlog_ln - snapshot_test.TestArchiveCommitlog
assert (
time.time() <= stop_time), "It's been over a {s}s and we haven't written a new " + \
> "commitlog segment. Something is wrong.".format(s=timeout)
E AssertionError: It's been over a {s}s and we haven't written a new commitlog segment. Something is wrong.
tools/hacks.py:61: AssertionError
12) test_archive_commitlog_with_active_commitlog - snapshot_test.TestArchiveCommitlog
assert (
time.time() <= stop_time), "It's been over a {s}s and we haven't written a new " + \
> "commitlog segment. Something is wrong.".format(s=timeout)
E AssertionError: It's been over a {s}s and we haven't written a new commitlog segment. Something is wrong.
13) test_dont_archive_commitlog - snapshot_test.TestArchiveCommitlog
assert (
time.time() <= stop_time), "It's been over a {s}s and we haven't written a new " + \
> "commitlog segment. Something is wrong.".format(s=timeout)
E AssertionError: It's been over a {s}s and we haven't written a new commitlog segment. Something is wrong.
tools/hacks.py:61: AssertionError
14) test_closing_connections - thrift_hsha_test.TestThriftHSHA
ccmlib.node.ToolError: Subprocess ['nodetool', '-h', 'localhost', '-p', '7100', 'enablethrift'] exited with non-zero status; exit status: 2; stderr: error: Could not create ServerSocket on address /127.0.0.1:9160. -- StackTrace -- org.apache.thrift.transport.TTransportException: Could not create ServerSocket on address /127.0.0.1:9160. at org.apache.thrift.transport.TNonblockingServerSocket.<init>(TNonblockingServerSocket.java:96) at org.apache.thrift.transport.TNonblockingServerSocket.<init>(TNonblockingServerSocket.java:79) at org.apache.thrift.transport.TNonblockingServerSocket.<init>(TNonblockingServerSocket.java:75) at org.apache.cassandra.thrift.TCustomNonblockingServerSocket.<init>(TCustomNonblockingServerSocket.java:39) at org.apache.cassandra.thrift.THsHaDisruptorServer$Factory.buildTServer(THsHaDisruptorServer.java:80) at org.apache.cassandra.thrift.TServerCustomFactory.buildTServer(TServerCustomFactory.java:55) at org.apache.cassandra.thrift.ThriftServer$ThriftServerThread.<init>(ThriftServer.java:131) at org.apache.cassandra.thrift.ThriftServer.start(ThriftServer.java:58) at org.apache.cassandra.service.StorageService.startRPCServer(StorageService.java:355) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:71) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:275) at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112) at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46) at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237) at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138) at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819) at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801) at javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1468) at javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:76) at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1309) at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1401) at javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:829) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:357) at sun.rmi.transport.Transport$1.run(Transport.java:200) at sun.rmi.transport.Transport$1.run(Transport.java:197) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.Transport.serviceCall(Transport.java:196) at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:568) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:683) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&&
3.0 :: extras from additional runs
1) test_decommissioned_node_cant_rejoin - topology_test.TestTopology
> assert re.search(rejoin_err,
'\n'.join(['\n'.join(err_list) for err_list in node3.grep_log_for_errors()]), re.MULTILINE)
E AssertionError: assert None
E + where None = <function search at 0x7fa9540c4ea0>('This node was decommissioned and will not rejoin the ring', '', <RegexFlag.MULTILINE: 8>)
E + where <function search at 0x7fa9540c4ea0> = re.search
E + and '' = <built-in method join of str object at 0x7fa9553f1ea0>([])
E + where <built-in method join of str object at 0x7fa9553f1ea0> = '\n'.join
E + and <RegexFlag.MULTILINE: 8> = re.MULTILINE
topology_test.py:396: AssertionError
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
3.0.16(-tentative-2) run :: https://circleci.com/workflow-run/5a1df5a1-f0c1-4ab4-a7db-e5551e7a5d38
no-vnodes:: 25 failures
- superset of current runs
- TestConsistency
- TestMaterializedViews
vnodes:: 22 failures
- superset of current runs
- same as no-vnodes basically
3.11.2(-tentative-2) run :: https://circleci.com/workflow-run/a4369ab0-ae11-497a-8e10-de3995d10f25
vnodes:: 18 failures
- TestConsistency
- TestMaterializedViews
- test_failure_threshold_deletions - paging_test.TestPagingWithDeletions
- TestGossiper
- TestArchiveCommitlog
no-vnodes:: 21 failurs
- TestCqlTracing
- TestDeleteInsert
- TestBaseSStableLoader
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment