This will require rebuilding the ledger for that validator. You will need enough disk space to download/rebuild that ledger.
# This is an example config for setting up a validator.
# see https://www.stellar.org/developers/stellar-core/learn/admin.html
# for how to properly configure your environment
# run `stellar-core --genseed` to generate a public key and secret seed.
# Let us know the public key so we can add you to the validator list.
# set NODE_SEED below to the secret seed generated above.
# uncomment those two lines if you are running a validator node
# NODE_SEED="S123456ABCDE"
# NODE_IS_VALIDATOR=true
LOG_FILE_PATH="stellar-prodex.log"
BUCKET_DIR_PATH="buckets-prodex"
DATABASE="sqlite3://stellar-prodex.db"
#FAILURE_SAFETY is minimum number of nodes that are allowed to fail before you no longer have quorum
FAILURE_SAFETY=1
CATCHUP_RECENT=100
NETWORK_PASSPHRASE="Public Global Stellar Network ; September 2015"
# Populate NODE_NAMES, KNOW_PEERS, QUORUM and HISTORY sections with information
# on other validators
# for example https://github.com/stellar/docs/blob/master/validators.md
# or from https://dashboard.stellar.org/
NODE_NAMES=[
"GDIQKLQVOCD5UD6MUI5D5PTPVX7WTP5TAPP5OBMOLENBBD5KG434KYQ2 stronghold1",
"GAOO3LWBC4XF6VWRP5ESJ6IBHAISVJMSBTALHOQM2EZG7Q477UWA6L7U eno",
"GCJCSMSPIWKKPR7WEPIQG63PDF7JGGEENRC33OKVBSPUDIRL6ZZ5M7OO tempo.eu.com",
"GC5SXLNAM3C4NMGK2PXK4R34B5GNZ47FYQ24ZIBFDFOCU6D4KBN4POAE satoshipay",
"GD7FVHL2KUTUYNOJFRUUDJPDRO2MAZJ5KP6EBCU6LKXHYGZDUFBNHXQI umbrel",
"GCGB2S2KGYARPVIA37HYZXVRM2YZUEXA6S33ZU5BUDC6THSB62LZSTYH sdf_watcher1",
"GCM6QMP3DLRPTAZW2UZPCPX2LF3SXWXKPMP3GKFZBDSF3QZGV2G5QSTK sdf_watcher2",
"GABMKJM6I25XI4K7U6XWMULOUQIQ27BCTMLS6BYYSOWKTBUXVRJSXHYQ sdf_watcher3",
]
KNOWN_PEERS=[
"core-live-a.stellar.org:11625",
"core-live-b.stellar.org:11625",
"core-live-c.stellar.org:11625",
"validator1.stellar.stronghold.co",
"stellar.256kw.com",
"stellar1.tempo.eu.com",
"stellar.satoshipay.io"
]
UNSAFE_QUORUM=true
# full validators (with history archive)
[QUORUM_SET]
THRESHOLD_PERCENT=50
VALIDATORS=[
"$sdf_watcher1", "$sdf_watcher2", "$sdf_watcher3"
]
# other validators that you want to include
# for best result, use a number of validators
# that can be expressed as 3f+1 (4, 7, 10 ,...)
[QUORUM_SET.basic]
VALIDATORS=[
"$stronghold1", "$eno", "$tempo.eu.com", "$satoshipay"
]
# History archives
# Stellar.org history store
[HISTORY.sdf1]
get="curl -sf http://history.stellar.org/prd/core-live/core_live_001/{0} -o {1}"
[HISTORY.sdf2]
get="curl -sf http://history.stellar.org/prd/core-live/core_live_002/{0} -o {1}"
[HISTORY.sdf3]
get="curl -sf http://history.stellar.org/prd/core-live/core_live_003/{0} -o {1}"
You will need to know the ledger number where you are "stuck". Typically this is the last entry from the logs before the problem started to happen:
2018-09-12T11:10:31.885 GC5SX [Ledger INFO] Closed ledger: [seq=19948007, hash=155198]
In this example, we're going to reconstruct ledger 19948007
Your logs will contain things like this:
2018-09-12T13:44:33.253 GC5SX [Process WARNING] process 24 exited 22: curl -sf http://history.stellar.org/prd/core-live/core_live_001/bucket/3e/44/d2/bucket-3e44d29cf75b2ceae1dcd58cbb18791c3ab998a89c848b231228db19bcf9c780.xdr.gz -o /data/buckets/tmp/repair-buckets-0cfc4c003dfba72b/bucket-3e44d29cf75b2ceae1dcd58cbb18791c3ab998a89c848b231228db19bcf9c780.xdr.gz.tmp
Take note of the hash of the bucket missing, in this example 3e44d29cf75b2ceae1dcd58cbb18791c3ab998a89c848b231228db19bcf9c780
There might be more missing.
- Initialize the temporary database:
./stellar-core --conf prod-example-docs.cfg --newdb - Trigger a manual catchup:
./stellar-core -conf prod-example-docs.cfg --catchup-at 19948006
when this completes, you can look in the buckets folder buckets-prodex:
find buckets-prodex -name 3e44d29cf75b2ceae1dcd58cbb18791c3ab998a89c848b231228db19bcf9c780
You may have to advance one ledger at a time:
- Trigger a manual catchup:
./stellar-core -conf prod-example-docs.cfg --catchup-at 19948007