Solana Testnet Restart - 27 May 2023.md

IMPORTANT: CLUSTER HAS RESTARTED

THESE INSTRUCTIONS ARE NOW OBSOLETE

Please use your common service file if network already started.

INSTALL THE LATEST SOLANA RELEASE AND ENSURE YOU HAVE THE UPDATED EXPECTED SHRED VERSION:

--expected-shred-version 28353 \

IMPORTANT: Instruction is valid only during restart.

Testnet 1.14.18 Restart

(If you are running Solana v1.14.17 or earlier, please upgrade to current recommended version 1.14.18 https://github.com/solana-labs/solana/releases/tag/v1.14.18)

The restart will start from the highest optimistic slot.

$ solana-ledger-tool -l /path_to_ledger/ latest-optimistic-slots
199641374 9GKQeYuVDxaGx34ABgZpJb1kzKdba9UQzvMX7xKcqTCp    2023-05-27T12:46:07.315+00:00

Note: If your last optimistic slot is lower than that the one listed below, this is likely your node crashed before it was able to observe the latest supermajority. Follow the instructions in the appendix

The known validator arguments provided below are validators who have the last optimistic slot and are providing snapshots for it, don’t keep them in your list as known validators after the restart as these are community validators not Solana Foundation nodes.

Important: DO NOT delete your ledger directory

Step 0: Stop the validator service

IMPORTANT: ⚠️MAKE SURE TO STOP YOUR VALIDATOR BEFORE DOING ANYTHING ELSE.⚠️

IMPORTANT: ⚠️DO NOT delete your ledger directory.⚠️

NOTE: Upgrade to v1.14.18 if you are running older Solana version

Step 1: Create a snapshot at slot 199641374 with some magic

This document assumes your ledger directory is called ledger/. If not then adjust the following commands accordingly.

Use the ledger tool to create a new snapshot at slot 199641374, replacing the two instances of to your actual ledger path:

$ solana-ledger-tool --ledger <ledger path> create-snapshot 199641374 <ledger path> --hard-fork 199641374

Or else, if you use separate directory for snapshots:

$ solana-ledger-tool --ledger <ledger path> --snapshot-archive-path <snapshots path> create-snapshot 199641374 <snapshots path> --hard-fork 199641374

Add --snapshot-archive-path before “create-snapshot” if you have a separate snapshot dir.

The final line of output should be “Shred version: 28353”, and this snapshot file should now exist: ledger/snapshot-199641374-njbdgtFvyDqiaCXRsgddu3J5uEtG6zPH81cmArcQSxS.tar.zst

Check your ledger/ directory to ensure that you have no snapshot newer than ledger/snapshot-199641374-..... This is very unlikely, but if found should be removed. Snapshots older than ledger/snapshot-199641374-.. should not be removed.

NOTE: If you receive “Error: Slot 199641374 is not available”, please see appendix

Step 2: Adjust your validator command-line arguments, temporarily for this restart to include:

--wait-for-supermajority 199641374 \
--no-snapshot-fetch \
--no-genesis-fetch \
--expected-genesis-hash 4uhcVJyU9pJkvQyS88uRDiswHXSCkY3zQawwpjk2NsNY \
--expected-bank-hash 7qKeiLLRxvxkD2hZx1Ai1RABKMg7x2n4GVJFcrEh3js2 \
--expected-shred-version 28353 \
--entrypoint entrypoint.testnet.solana.com:8001 \
--entrypoint entrypoint2.testnet.solana.com:8001 \
--entrypoint entrypoint3.testnet.solana.com:8001 \
--entrypoint entrypoint.testnet.solana.sergo.dev:8001 \
--known-validator eoKpUABi59aT4rR9HGS3LcMecfut9x7zJyodWWP43YQ \
--known-validator GAPNvBD6MXboQmxP9XTCC4CMsT5gUpdFZWbnj4Tz2s7i \
--known-validator 5D1fNXzvv5NjV1ysLjirC4WY92RNsVH18vjmcszZd8on \
--known-validator BFquPCAYdjN9QyLVfuGrQdJTF9Ct7Z85FDxhFeLcpFqR \
--known-validator 9e2RvEzemWs6ZkEhdW2NddSWiFKgJfkw5LWGtgwvPnvw \

(Remove the previous value of “--expected-shred-version“ if present)

Once the cluster restarts and normal operation resumes, remember to remove these arguments. They are only required for the restart.

Step 3: Start your validator

As it boots, it will load the snapshot for slot 199641374 and wait for 80% of the stake to come online before producing/validating new blocks.

To confirm your restarted validator is correctly waiting for 80% stake, look for this periodic log message to confirm it is waiting: INFO solana_core::validator] Waiting for 80% of activated stake at slot 199641374 to be in gossip...

And if you have RPC enabled, ask it repeated for the current slot:

$ solana --url http://127.0.0.1:8899 slot

Any number other than 199641374 means you did not complete the steps correctly.

Appendix: Resolution if you did not preserve your ledger or your last optimistic slot is below 199641374

NOT RECOMMENDED - this resolution should only be attempted if your ledger/ directory is unavailable.

If your ledger history is corrupt or otherwise unavailable, clean up your ledger/ directory instead of Steps 1 & 2, then complete Steps 3.

OBS: remove --no-snapshot-fetch if set

Add these arguments to restart:

--wait-for-supermajority 199641374 \
--expected-shred-version 28353 \
--entrypoint entrypoint.testnet.solana.com:8001 \
--entrypoint entrypoint2.testnet.solana.com:8001 \
--entrypoint entrypoint3.testnet.solana.com:8001 \
--entrypoint entrypoint.testnet.solana.sergo.dev:8001 \
--expected-genesis-hash 4uhcVJyU9pJkvQyS88uRDiswHXSCkY3zQawwpjk2NsNY \
--expected-bank-hash 7qKeiLLRxvxkD2hZx1Ai1RABKMg7x2n4GVJFcrEh3js2 \
--known-validator eoKpUABi59aT4rR9HGS3LcMecfut9x7zJyodWWP43YQ \
--known-validator GAPNvBD6MXboQmxP9XTCC4CMsT5gUpdFZWbnj4Tz2s7i \
--known-validator 5D1fNXzvv5NjV1ysLjirC4WY92RNsVH18vjmcszZd8on \
--known-validator BFquPCAYdjN9QyLVfuGrQdJTF9Ct7Z85FDxhFeLcpFqR \
 --known-validator 9e2RvEzemWs6ZkEhdW2NddSWiFKgJfkw5LWGtgwvPnvw \

If you download snapshot manually, please go to Step 2, not Appendix!

Links to snapshot(different locations):

wget --trust-server-names http://107.155.91.150:8899/snapshot.tar.bz2
wget --trust-server-names http://145.40.109.3:8899/snapshot.tar.bz2
wget --trust-server-names http://testnet.solana.sergo.dev:8899/snapshot.tar.bz2
http://69.197.4.3:6969/snapshot-199641374-njbdgtFvyDqiaCXRsgddu3J5uEtG6zPH81cmArcQSxS.tar.zst
http://69.197.42.30:6969/snapshot-199641374-njbdgtFvyDqiaCXRsgddu3J5uEtG6zPH81cmArcQSxS.tar.zst

Genesis:

 wget http://api.testnet.solana.com/genesis.tar.bz2