Last active
June 30, 2021 18:38
-
-
Save PeterCorless/4d85919f9f63dae7aa9d75704f3b4a4f to your computer and use it in GitHub Desktop.
Scylla-Manager-Cloning
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| $ sctool status | |
| Cluster: test (77472fd3-a85f-4c8e-bdf5-6c17696ab975) | |
| Datacenter: dc1 | |
| ╭────┬────────────┬──────────┬──────────┬────────────────┬──────────┬──────┬──────────┬────────┬──────────┬──────────────────────────────────────╮ | |
| │ │ Alternator │ CQL │ REST │ Address │ Uptime │ CPUs │ Memory │ Scylla │ Agent │ Host ID │ | |
| ├────┼────────────┼──────────┼──────────┼────────────────┼──────────┼──────┼──────────┼────────┼──────────┼──────────────────────────────────────┤ | |
| │ UN │ UP (13ms) │ UP (5ms) │ UP (2ms) │ 192.168.100.11 │ 7h26m34s │ 4 │ 31.12GiB │ 4.2.1 │ Snapshot │ 3827c1d1-b300-4093-9839-0cf01f1b9346 │ | |
| │ UN │ UP (9ms) │ UP (3ms) │ UP (2ms) │ 192.168.100.12 │ 7h26m34s │ 4 │ 31.12GiB │ 4.2.1 │ Snapshot │ a4f20c78-0a64-490c-b7f4-f52a62667ff8 │ | |
| │ UN │ UP (14ms) │ UP (4ms) │ UP (2ms) │ 192.168.100.13 │ 7h26m34s │ 4 │ 31.12GiB │ 4.2.1 │ Snapshot │ ffa39462-d3f1-46d9-875c-ef89044e951e │ | |
| ╰────┴────────────┴──────────┴──────────┴────────────────┴──────────┴──────┴──────────┴────────┴──────────┴───────────────────────────────────── |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| $ scylla-manager-agent download-files -L s3:manager-test-demo1 --list-nodes | |
| Cluster: prod (9e9b392b-fdd8-4b36-8980-d937949bf6ce) | |
| AWS_EU_CENTRAL_1: | |
| - 18.194.132.192 (3dda17c5-de2a-4bad-80cd-d695c328601a) | |
| - 18.197.114.43 (b0b60778-f4cd-4d77-857f-5a076cfa938b) | |
| - 3.66.107.199 (aeca2b65-8e0c-4158-9886-3ffa54451d2f) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| host_id: | |
| "35.157.153.136": 3dda17c5-de2a-4bad-80cd-d695c328601a | |
| "3.68.171.110": b0b60778-f4cd-4d77-857f-5a076cfa938b | |
| "18.192.18.219": aeca2b65-8e0c-4158-9886-3ffa54451d2 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| $ scylla-manager-agent download-files -L s3:manager-test-demo1 --list-snapshots -n 3dda17c5-de2a-4bad-80cd-d695c328601a | |
| sm_20210624122942UTC |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| $ cat vars.yaml | |
| # backup_location specifies the location parameter used in Scylla Manager | |
| # when scheduling a backup of a cluster. | |
| backup_location: s3:manager-test-demo1 | |
| # snapshot_tag specifies the Scylla Manager snapshot tag you want to restore. | |
| snapshot_tag: sm_20210624122942UTC | |
| # host_id specifies a mapping from the clone cluster node IP to the source | |
| # cluster host IDs. | |
| host_id: | |
| "35.157.153.136": 3dda17c5-de2a-4bad-80cd-d695c328601a | |
| "3.68.171.110": b0b60778-f4cd-4d77-857f-5a076cfa938b | |
| "18.192.18.219": aeca2b65-8e0c-4158-9886-3ffa54451d2f |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| $ cat hosts | |
| 35.157.153.136 | |
| 3.68.171.110 | |
| 18.192.18.219 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| $ sudo -u scylla scylla-manager-agent download-files -L s3:manager-test-demo1 -n aeca2b65-8e0c-4158-9886-3ffa54451d2f -d /var/lib/scylla/data/ --dry-run -T sm_20210624122942UTC | |
| Cluster: prod (9e9b392b-fdd8-4b36-8980-d937949bf6ce) | |
| Datacenter: AWS_EU_CENTRAL_1 | |
| Node: 3.66.107.199 (aeca2b65-8e0c-4158-9886-3ffa54451d2f) | |
| Time: 2021-06-24 12:29:42 +0000 UTC | |
| Size: 324.054G | |
| Download: | |
| - system_auth.role_attributes (4.933k) to /var/lib/scylla/data/system_auth/role_attributes-6b8c7359a84333f2a1d85dc6a187436f | |
| - system_auth.role_members (4.914k) to /var/lib/scylla/data/system_auth/role_members-0ecdaa87f8fb3e6088d174fb36fe5c0d | |
| - system_auth.role_permissions (42.805k) to /var/lib/scylla/data/system_auth/role_permissions-3afbe79f219431a7add7f5ab90d8ec9c | |
| - system_auth.roles (47.932k) to /var/lib/scylla/data/system_auth/roles-5bc52802de2535edaeab188eecebb090 | |
| - keyspace1.data_0 (40.431G) to /var/lib/scylla/data/keyspace1/data_0-5fb8c700d4e011ebbfe5000000000001 | |
| - keyspace1.data_1 (40.448G) to /var/lib/scylla/data/keyspace1/data_1-5fb89ff0d4e011ebb034000000000002 | |
| - keyspace1.data_2 (40.461G) to /var/lib/scylla/data/keyspace1/data_2-5fb8c702d4e011ebbfe5000000000001 | |
| - keyspace1.data_3 (40.435G) to /var/lib/scylla/data/keyspace1/data_3-5fb8c700d4e011eb8930000000000003 | |
| - keyspace1.data_4 (40.954G) to /var/lib/scylla/data/keyspace1/data_4-5fab5980d4e011eb9c15000000000001 | |
| - keyspace1.data_5 (40.440G) to /var/lib/scylla/data/keyspace1/data_5-5faba7a0d4e011eba8f8000000000003 | |
| - keyspace1.data_6 (40.428G) to /var/lib/scylla/data/keyspace1/data_6-619f8860d4e011eb850d000000000001 | |
| - keyspace1.data_7 (40.456G) to /var/lib/scylla/data/keyspace1/data_7-61a072c0d4e011eb8930000000000003 | |
| - system_schema.aggregates (10.533k) to /var/lib/scylla/data/system_schema/aggregates-924c55872e3a345bb10c12f37c1ba895 | |
| - system_schema.columns (108.946k) to /var/lib/scylla/data/system_schema/columns-24101c25a2ae3af787c1b40ee1aca33f | |
| - system_schema.computed_columns (10.271k) to /var/lib/scylla/data/system_schema/computed_columns-cc7c7069374033c192a4c3de78dbd2c4 | |
| - system_schema.dropped_columns (10.547k) to /var/lib/scylla/data/system_schema/dropped_columns-5e7583b5f3f43af19a39b7e1d6f5f11f | |
| - system_schema.functions (10.723k) to /var/lib/scylla/data/system_schema/functions-96489b7980be3e14a70166a0b9159450 | |
| - system_schema.indexes (10.600k) to /var/lib/scylla/data/system_schema/indexes-0feb57ac311f382fba6d9024d305702f | |
| - system_schema.keyspaces (98.598k) to /var/lib/scylla/data/system_schema/keyspaces-abac5682dea631c5b535b3d6cffd0fb6 | |
| - system_schema.scylla_tables (100.185k) to /var/lib/scylla/data/system_schema/scylla_tables-5d912ff1f7593665b2c88042ab5103dd | |
| - system_schema.tables (107.300k) to /var/lib/scylla/data/system_schema/tables-afddfb9dbc1e30688056eed6c302ba09 | |
| - system_schema.triggers (10.203k) to /var/lib/scylla/data/system_schema/triggers-4df70b666b05325195a132b54005fd48 | |
| - system_schema.types (10.322k) to /var/lib/scylla/data/system_schema/types-5a8b1ca866023f77a0459273d308917a | |
| - system_schema.view_virtual_columns (10.369k) to /var/lib/scylla/data/system_schema/view_virtual_columns-08843b6345dc3be29798a0418295cfaa | |
| - system_schema.views (13.039k) to /var/lib/scylla/data/system_schema/views-9786ac1cdd583201a7cdad556410c985 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| $ cat ~/.ansible.cfg | |
| [defaults] | |
| remote_user = support | |
| private_key_file = /path/to/aws/pem/file |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| ansible-playbook -i hosts -e @vars.yaml restore.yaml | |
| PLAY [Restore] ********************************************************************************************************************************************************************************************************************************************************** | |
| TASK [Stop Scylla service] ********************************************************************************************************************************************************************************************************************************************** | |
| changed: [18.192.18.219] | |
| changed: [35.157.153.136] | |
| changed: [3.68.171.110] | |
| TASK [Remove Scylla directories] **************************************************************************************************************************************************************************************************************************************** | |
| changed: [3.68.171.110] => (item=data) | |
| changed: [35.157.153.136] => (item=data) | |
| changed: [18.192.18.219] => (item=data) | |
| changed: [3.68.171.110] => (item=commitlog) | |
| changed: [18.192.18.219] => (item=commitlog) | |
| changed: [35.157.153.136] => (item=commitlog) | |
| changed: [3.68.171.110] => (item=hints) | |
| changed: [18.192.18.219] => (item=hints) | |
| changed: [35.157.153.136] => (item=hints) | |
| changed: [3.68.171.110] => (item=view_hints) | |
| changed: [18.192.18.219] => (item=view_hints) | |
| changed: [35.157.153.136] => (item=view_hints) | |
| TASK [Recreate Scylla directories] ************************************************************************************************************************************************************************************************************************************** | |
| changed: [3.68.171.110] => (item=data) | |
| changed: [18.192.18.219] => (item=data) | |
| changed: [35.157.153.136] => (item=data) | |
| changed: [3.68.171.110] => (item=commitlog) | |
| changed: [18.192.18.219] => (item=commitlog) | |
| changed: [35.157.153.136] => (item=commitlog) | |
| changed: [3.68.171.110] => (item=hints) | |
| changed: [18.192.18.219] => (item=hints) | |
| changed: [35.157.153.136] => (item=hints) | |
| changed: [3.68.171.110] => (item=view_hints) | |
| changed: [18.192.18.219] => (item=view_hints) | |
| changed: [35.157.153.136] => (item=view_hints) | |
| TASK [Get initial token] ************************************************************************************************************************************************************************************************************************************************ | |
| changed: [18.192.18.219] | |
| changed: [3.68.171.110] | |
| changed: [35.157.153.136] | |
| TASK [Set initial_token in scylla.yaml] ********************************************************************************************************************************************************************************************************************************* | |
| changed: [3.68.171.110] | |
| changed: [35.157.153.136] | |
| changed: [18.192.18.219] | |
| TASK [Get seeds list] *************************************************************************************************************************************************************************************************************************************************** | |
| changed: [3.68.171.110] | |
| changed: [35.157.153.136] | |
| changed: [18.192.18.219] | |
| TASK [Temporarily make each node a seed] ******************************************************************************************************************************************************************************************************************************** | |
| ok: [3.68.171.110] | |
| ok: [35.157.153.136] | |
| ok: [18.192.18.219] | |
| TASK [Start Scylla service] ********************************************************************************************************************************************************************************************************************************************* | |
| changed: [18.192.18.219] | |
| changed: [3.68.171.110] | |
| changed: [35.157.153.136] | |
| TASK [Wait for CQL port] ************************************************************************************************************************************************************************************************************************************************ | |
| ok: [18.192.18.219] | |
| ok: [35.157.153.136] | |
| ok: [3.68.171.110] | |
| TASK [Stop Scylla service] ********************************************************************************************************************************************************************************************************************************************** | |
| changed: [18.192.18.219] | |
| changed: [3.68.171.110] | |
| changed: [35.157.153.136] | |
| TASK [Temporarily make the first node a seed] *************************************************************************************************************************************************************************************************************************** | |
| changed: [3.68.171.110] | |
| changed: [18.192.18.219] | |
| changed: [35.157.153.136] | |
| TASK [Download data] **************************************************************************************************************************************************************************************************************************************************** | |
| [WARNING]: Module remote_tmp /var/lib/scylla/.ansible/tmp did not exist and was created with a mode of 0700, this may cause issues when running as another user. To avoid this, create the remote_tmp dir with the correct permissions manually | |
| changed: [18.192.18.219] | |
| changed: [3.68.171.110] | |
| changed: [35.157.153.136] | |
| TASK [Start Scylla service in the seed] ********************************************************************************************************************************************************************************************************************************* | |
| skipping: [3.68.171.110] | |
| skipping: [18.192.18.219] | |
| changed: [35.157.153.136] | |
| TASK [Start Scylla service] ********************************************************************************************************************************************************************************************************************************************* | |
| ok: [35.157.153.136] | |
| changed: [3.68.171.110] | |
| changed: [18.192.18.219] | |
| TASK [Wait for CQL port] ************************************************************************************************************************************************************************************************************************************************ | |
| ok: [35.157.153.136] | |
| ok: [3.68.171.110] | |
| ok: [18.192.18.219] | |
| TASK [Resore seeds list] ************************************************************************************************************************************************************************************************************************************************ | |
| changed: [18.192.18.219] | |
| changed: [3.68.171.110] | |
| changed: [35.157.153.136] | |
| TASK [Restart Scylla service] ******************************************************************************************************************************************************************************************************************************************* | |
| changed: [35.157.153.136] | |
| changed: [3.68.171.110] | |
| changed: [18.192.18.219] | |
| PLAY RECAP ************************************************************************************************************************************************************************************************************************************************************** | |
| 18.192.18.219 : ok=16 changed=13 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0 | |
| 3.68.171.110 : ok=16 changed=13 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0 | |
| 35.157.153.136 : ok=17 changed=13 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Connected to 9b71aa77-be72-496f-bf14-c159fa82a84b at 127.0.0.1:9042. | |
| [cqlsh 5.0.1 | Cassandra 3.0.8 | CQL spec 3.3.1 | Native protocol v4] | |
| Use HELP for help. | |
| cqlsh> SELECT * FROM | |
| keyspace1. system. system_auth. system_distributed. system_schema. system_traces. | |
| cqlsh> SELECT * FROM keyspace1.data_ | |
| data_0 data_1 data_2 data_3 data_4 data_5 data_6 data_7 | |
| cqlsh> SELECT id FROM keyspace1.data_0 LIMIT 10; | |
| id | |
| -------------------------------------- | |
| b76af606-e6fa-4001-8660-a7c86ea0f545 | |
| 71187315-5dd7-4df6-9fc3-324aee6b6456 | |
| a19d1824-6301-41ca-97b9-c85200e8f070 | |
| 5d29fd1c-cda3-4efe-b6d7-96061c2a5257 | |
| e4c1411b-7de5-45d6-b375-d02332f0394c | |
| 4b3ca60a-b1e3-410d-ad7a-27c9ab22734a | |
| a302b84c-3da3-468c-a628-28000dd5cc7f | |
| debb2d34-f107-41c1-b3e1-49feed15982b | |
| ba660b34-d2b1-4e1e-859a-f9bc66617ca6 | |
| 93a39da5-c23b-4590-952c-1e86a09d60a9 | |
| (10 rows) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment