Created
April 9, 2024 22:06
-
-
Save rustyrazorblade/2afb78bb092f0202dd8e2413fbb44e47 to your computer and use it in GitHub Desktop.
read repair demo
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ubuntu@cassandra0:~$ nt status | |
| Datacenter: us-west-2 | |
| ===================== | |
| Status=Up/Down | |
| |/ State=Normal/Leaving/Joining/Moving | |
| -- Address Load Tokens Owns (effective) Host ID Rack | |
| UN 172.31.29.91 68.79 KiB 4 100.0% aaaaf5d0-4f00-48e4-8530-6bb7f951a0d8 us-west-2b | |
| UN 172.31.47.193 68.8 KiB 4 100.0% a7399471-7c40-4b00-aa94-0f94594c7cff us-west-2a | |
| ubuntu@cassandra0:~$ c | |
| Connected to test at cassandra0:9042 | |
| [cqlsh 6.0.0 | Cassandra 4.0.12 | CQL spec 3.4.5 | Native protocol v5] | |
| Use HELP for help. | |
| cqlsh> create KEYSPACE test WITH replication = {'class': 'SimpleStrategy', 'replication_factor': 2}; | |
| cqlsh> use test; | |
| cqlsh:test> create table blah (id int primary key, foo text); | |
| cqlsh:test> insert into blah (id, foo) values (1, 'data'); | |
| cqlsh:test> exit | |
| # locally, c-all to flush on all hosts | |
| ❯ c-all nodetool flush | |
| ubuntu@cassandra0:~$ c | |
| Connected to test at cassandra0:9042 | |
| [cqlsh 6.0.0 | Cassandra 4.0.12 | CQL spec 3.4.5 | Native protocol v5] | |
| Use HELP for help. | |
| cqlsh> select * from test.blah; | |
| id | foo | |
| ----+------ | |
| 1 | data | |
| (1 rows) | |
| cqlsh> exit | |
| ubuntu@cassandra0:~$ sudo systemctl stop cassandra | |
| sudo rm -rf /mnt/cassandra/data/test/blah-9e90ac30f6bc11eeac6df3d98ab66ac9/* | |
| ubuntu@cassandra0:~$ nt status | |
| Datacenter: us-west-2 | |
| ===================== | |
| Status=Up/Down | |
| |/ State=Normal/Leaving/Joining/Moving | |
| -- Address Load Tokens Owns (effective) Host ID Rack | |
| UN 172.31.29.91 127.51 KiB 4 100.0% aaaaf5d0-4f00-48e4-8530-6bb7f951a0d8 us-west-2b | |
| UN 172.31.47.193 117.92 KiB 4 100.0% a7399471-7c40-4b00-aa94-0f94594c7cff us-west-2a | |
| ubuntu@cassandra0:~$ | |
| ubuntu@cassandra0:~$ | |
| ubuntu@cassandra0:~$ c | |
| Connected to test at cassandra0:9042 | |
| [cqlsh 6.0.0 | Cassandra 4.0.12 | CQL spec 3.4.5 | Native protocol v5] | |
| Use HELP for help. | |
| cqlsh> tracing on; | |
| Now Tracing is enabled | |
| cqlsh> select * from test.blah where id = 1; | |
| id | foo | |
| ----+----- | |
| (0 rows) | |
| Tracing session: 2ba6fd40-f6bd-11ee-bed8-6be987d74d73 | |
| activity | timestamp | source | source_elapsed | client | |
| -----------------------------------------------------------------------------+----------------------------+---------------+----------------+--------------- | |
| Execute CQL3 query | 2024-04-09 22:04:43.669000 | 172.31.47.193 | 0 | 172.31.47.193 | |
| Parsing select * from test.blah where id = 1; [Native-Transport-Requests-1] | 2024-04-09 22:04:43.672000 | 172.31.47.193 | 2991 | 172.31.47.193 | |
| Preparing statement [Native-Transport-Requests-1] | 2024-04-09 22:04:43.673000 | 172.31.47.193 | 3799 | 172.31.47.193 | |
| Executing single-partition query on blah [ReadStage-2] | 2024-04-09 22:04:43.675000 | 172.31.47.193 | 5761 | 172.31.47.193 | |
| Acquiring sstable references [ReadStage-2] | 2024-04-09 22:04:43.675000 | 172.31.47.193 | 5932 | 172.31.47.193 | |
| Merging memtable contents [ReadStage-2] | 2024-04-09 22:04:43.675000 | 172.31.47.193 | 6058 | 172.31.47.193 | |
| Read 0 live rows and 0 tombstone cells [ReadStage-2] | 2024-04-09 22:04:43.675000 | 172.31.47.193 | 6317 | 172.31.47.193 | |
| Request complete | 2024-04-09 22:04:43.675946 | 172.31.47.193 | 6946 | 172.31.47.193 | |
| cqlsh> consistency quorum; | |
| Consistency level set to QUORUM. | |
| cqlsh> select * from test.blah where id = 1; | |
| id | foo | |
| ----+------ | |
| 1 | data | |
| (1 rows) | |
| Tracing session: 3a165ce0-f6bd-11ee-bed8-6be987d74d73 | |
| activity | timestamp | source | source_elapsed | client | |
| -------------------------------------------------------------------------------------------------------------------------------------+----------------------------+---------------+----------------+--------------- | |
| Execute CQL3 query | 2024-04-09 22:05:07.886000 | 172.31.47.193 | 0 | 172.31.47.193 | |
| Parsing select * from test.blah where id = 1; [Native-Transport-Requests-1] | 2024-04-09 22:05:07.886000 | 172.31.47.193 | 407 | 172.31.47.193 | |
| Preparing statement [Native-Transport-Requests-1] | 2024-04-09 22:05:07.887000 | 172.31.47.193 | 858 | 172.31.47.193 | |
| reading digest from /172.31.29.91:7000 [Native-Transport-Requests-1] | 2024-04-09 22:05:07.888000 | 172.31.47.193 | 1757 | 172.31.47.193 | |
| Executing single-partition query on blah [ReadStage-2] | 2024-04-09 22:05:07.888000 | 172.31.47.193 | 2254 | 172.31.47.193 | |
| Acquiring sstable references [ReadStage-2] | 2024-04-09 22:05:07.888000 | 172.31.47.193 | 2442 | 172.31.47.193 | |
| Sending READ_REQ message to /172.31.29.91:7000 message size 80 bytes [Messaging-EventLoop-3-2] | 2024-04-09 22:05:07.889000 | 172.31.47.193 | 2711 | 172.31.47.193 | |
| Merging memtable contents [ReadStage-2] | 2024-04-09 22:05:07.892000 | 172.31.47.193 | 6663 | 172.31.47.193 | |
| Read 0 live rows and 0 tombstone cells [ReadStage-2] | 2024-04-09 22:05:07.893000 | 172.31.47.193 | 6933 | 172.31.47.193 | |
| READ_REQ message received from /172.31.47.193:7000 [Messaging-EventLoop-3-2] | 2024-04-09 22:05:07.895000 | 172.31.29.91 | 298 | 172.31.47.193 | |
| Executing single-partition query on blah [ReadStage-1] | 2024-04-09 22:05:07.898000 | 172.31.29.91 | 2803 | 172.31.47.193 | |
| Acquiring sstable references [ReadStage-1] | 2024-04-09 22:05:07.898000 | 172.31.29.91 | 2985 | 172.31.47.193 | |
| Merging memtable contents [ReadStage-1] | 2024-04-09 22:05:07.898000 | 172.31.29.91 | 3097 | 172.31.47.193 | |
| Partition index with 0 entries found for sstable 1 [ReadStage-1] | 2024-04-09 22:05:07.898000 | 172.31.29.91 | 3511 | 172.31.47.193 | |
| Read 1 live rows and 0 tombstone cells [ReadStage-1] | 2024-04-09 22:05:07.899000 | 172.31.29.91 | 4286 | 172.31.47.193 | |
| Enqueuing response to /172.31.47.193:7000 [ReadStage-1] | 2024-04-09 22:05:07.899000 | 172.31.29.91 | 4435 | 172.31.47.193 | |
| Sending READ_RSP message to /172.31.47.193:7000 message size 50 bytes [Messaging-EventLoop-3-1] | 2024-04-09 22:05:07.901000 | 172.31.29.91 | 6263 | 172.31.47.193 | |
| READ_RSP message received from /172.31.29.91:7000 [Messaging-EventLoop-3-1] | 2024-04-09 22:05:07.902000 | 172.31.47.193 | 16152 | 172.31.47.193 | |
| Processing response from /172.31.29.91:7000 [RequestResponseStage-2] | 2024-04-09 22:05:07.902000 | 172.31.47.193 | 16519 | 172.31.47.193 | |
| Digest mismatch: Mismatch for key DecoratedKey(-4069959284402364209, 00000001) [Native-Transport-Requests-1] | 2024-04-09 22:05:07.903000 | 172.31.47.193 | 16942 | 172.31.47.193 | |
| Enqueuing full data read to Full(/172.31.29.91:7000,(-6755555680146753369,-3864511298762678531]) [Native-Transport-Requests-1] | 2024-04-09 22:05:07.904000 | 172.31.47.193 | 18086 | 172.31.47.193 | |
| Sending READ_REQ message to /172.31.29.91:7000 message size 79 bytes [Messaging-EventLoop-3-2] | 2024-04-09 22:05:07.904000 | 172.31.47.193 | 18506 | 172.31.47.193 | |
| READ_REQ message received from /172.31.47.193:7000 [Messaging-EventLoop-3-2] | 2024-04-09 22:05:07.905000 | 172.31.29.91 | 11 | 172.31.47.193 | |
| Executing single-partition query on blah [ReadStage-3] | 2024-04-09 22:05:07.905000 | 172.31.47.193 | 19149 | 172.31.47.193 | |
| Executing single-partition query on blah [ReadStage-2] | 2024-04-09 22:05:07.905000 | 172.31.29.91 | 235 | 172.31.47.193 | |
| Acquiring sstable references [ReadStage-3] | 2024-04-09 22:05:07.905000 | 172.31.47.193 | 19335 | 172.31.47.193 | |
| Merging memtable contents [ReadStage-3] | 2024-04-09 22:05:07.905000 | 172.31.47.193 | 19534 | 172.31.47.193 | |
| Acquiring sstable references [ReadStage-2] | 2024-04-09 22:05:07.906000 | 172.31.29.91 | 348 | 172.31.47.193 | |
| Read 0 live rows and 0 tombstone cells [ReadStage-3] | 2024-04-09 22:05:07.906000 | 172.31.47.193 | 19823 | 172.31.47.193 | |
| Merging memtable contents [ReadStage-2] | 2024-04-09 22:05:07.906000 | 172.31.29.91 | 458 | 172.31.47.193 | |
| Key cache hit for sstable 1 [ReadStage-2] | 2024-04-09 22:05:07.906000 | 172.31.29.91 | 630 | 172.31.47.193 | |
| Read 1 live rows and 0 tombstone cells [ReadStage-2] | 2024-04-09 22:05:07.906000 | 172.31.29.91 | 1036 | 172.31.47.193 | |
| Enqueuing response to /172.31.47.193:7000 [ReadStage-2] | 2024-04-09 22:05:07.906000 | 172.31.29.91 | 1184 | 172.31.47.193 | |
| Sending READ_RSP message to /172.31.47.193:7000 message size 65 bytes [Messaging-EventLoop-3-1] | 2024-04-09 22:05:07.909000 | 172.31.29.91 | 3965 | 172.31.47.193 | |
| READ_RSP message received from /172.31.29.91:7000 [Messaging-EventLoop-3-1] | 2024-04-09 22:05:07.910000 | 172.31.47.193 | 23924 | 172.31.47.193 | |
| Processing response from /172.31.29.91:7000 [RequestResponseStage-3] | 2024-04-09 22:05:07.910000 | 172.31.47.193 | 24160 | 172.31.47.193 | |
| Sending read-repair-mutation to Full(/172.31.47.193:7000,(-6755555680146753369,-3864511298762678531]) [Native-Transport-Requests-1] | 2024-04-09 22:05:07.935000 | 172.31.47.193 | 49289 | 172.31.47.193 | |
| Executing single-partition query on peers_v2 [Native-Transport-Requests-1] | 2024-04-09 22:05:07.937000 | 172.31.47.193 | 51002 | 172.31.47.193 | |
| Acquiring sstable references [Native-Transport-Requests-1] | 2024-04-09 22:05:07.938000 | 172.31.47.193 | 51788 | 172.31.47.193 | |
| Merging memtable contents [Native-Transport-Requests-1] | 2024-04-09 22:05:07.938000 | 172.31.47.193 | 52632 | 172.31.47.193 | |
| Read 0 live rows and 0 tombstone cells [Native-Transport-Requests-1] | 2024-04-09 22:05:07.939000 | 172.31.47.193 | 53594 | 172.31.47.193 | |
| Sending READ_REPAIR_REQ message to /172.31.47.193:7000 message size 80 bytes [Messaging-EventLoop-3-1] | 2024-04-09 22:05:07.944000 | 172.31.47.193 | 58452 | 172.31.47.193 | |
| READ_REPAIR_REQ message received from /172.31.47.193:7000 [Messaging-EventLoop-3-2] | 2024-04-09 22:05:07.947000 | 172.31.47.193 | 61552 | 172.31.47.193 | |
| Appending to commitlog [MutationStage-2] | 2024-04-09 22:05:07.948000 | 172.31.47.193 | 61869 | 172.31.47.193 | |
| Adding to blah memtable [MutationStage-2] | 2024-04-09 22:05:07.948000 | 172.31.47.193 | 62146 | 172.31.47.193 | |
| Sending READ_REPAIR_RSP message to /172.31.47.193:7000 message size 33 bytes [Messaging-EventLoop-3-1] | 2024-04-09 22:05:07.949000 | 172.31.47.193 | 62799 | 172.31.47.193 | |
| READ_REPAIR_RSP message received from /172.31.47.193:7000 [Messaging-EventLoop-3-2] | 2024-04-09 22:05:07.951000 | 172.31.47.193 | 65300 | 172.31.47.193 | |
| Processing response from /172.31.47.193:7000 [RequestResponseStage-3] | 2024-04-09 22:05:07.952000 | 172.31.47.193 | 65862 | 172.31.47.193 | |
| Request complete | 2024-04-09 22:05:07.953888 | 172.31.47.193 | 67888 | 172.31.47.193 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment