Skip to content

Instantly share code, notes, and snippets.

@brianlong
Created December 15, 2020 03:21
Show Gist options
  • Save brianlong/534a85dc304bf94171c33c07220bb4e9 to your computer and use it in GitHub Desktop.
Save brianlong/534a85dc304bf94171c33c07220bb4e9 to your computer and use it in GitHub Desktop.
solana@solana-tds:~$ cat 71bhKKL89U3dNHzuZVZ7KarqV6XtHEgjXjvJTsguD11B.log | grep -B 20 -A 140 '52742164 leader: 71bhKKL89U3dNHzuZVZ7KarqV6XtHEgjXjvJTsguD11B'
[2020-12-15T01:14:53.673326033Z WARN solana_metrics::counter] COUNTER:{"name": "poh_recorder-record_ms", "counts": 1037622, "samples": 1034000, "now": 1607994893673, "events": 0}
[2020-12-15T01:14:53.673757860Z INFO solana_metrics::metrics] datapoint: best_slot slot=52742163i best_slot=52742163i
[2020-12-15T01:14:53.673776904Z INFO solana_metrics::metrics] datapoint: bank_weight slot=52742163i weight="0"
[2020-12-15T01:14:53.673759069Z INFO solana_core::replay_stage] 71bhKKL89U3dNHzuZVZ7KarqV6XtHEgjXjvJTsguD11B slot_weight: 52742163 0 0 52742162
[2020-12-15T01:14:53.673813457Z INFO solana_core::replay_stage] validator fork confirmed 52742161 1660ms
[2020-12-15T01:14:53.673828085Z INFO solana_core::replay_stage] validator fork confirmed 52742162 929ms
[2020-12-15T01:14:53.673830074Z INFO solana_metrics::metrics] datapoint: validator-confirmation duration_ms=1660i
[2020-12-15T01:14:53.673836850Z INFO solana_metrics::metrics] datapoint: validator-confirmation duration_ms=929i
[2020-12-15T01:14:53.673839337Z INFO solana_core::replay_stage] voting: 52742163 0
[2020-12-15T01:14:53.673847318Z INFO solana_core::replay_stage] LEADER CHANGE at slot: 52742163 leader: 3Wyqj2cgKYK2sSSb3wVv3wJ5yD3yigV8iLLttkZfKn8d. I am no longer the leader
[2020-12-15T01:14:53.673909855Z INFO solana_metrics::metrics] datapoint: tower-vote latest=52742163i root=52742124i
[2020-12-15T01:14:53.680943432Z INFO solana_metrics::metrics] datapoint: purge_slots_time remove_storages_elapsed=0i drop_storage_entries_elapsed=0i num_slots_removed=0i total_removed_storage_entries=0i total_removed_bytes=0i
[2020-12-15T01:14:53.681656767Z INFO solana_core::replay_stage] new root 52742124
[2020-12-15T01:14:53.682059925Z INFO solana_core::replay_stage] vote bank: Some((52742163, SameFork)) reset bank: 52742163
[2020-12-15T01:14:53.682138110Z INFO solana_metrics::metrics] datapoint: blocks_produced num_blocks_on_fork=534089i num_dropped_blocks_on_fork=110205i
[2020-12-15T01:14:53.682147592Z INFO solana_core::poh_recorder] reset poh from: 4EZoLtRnqyf7eUiWEsS2MXqT6s7nkSPiebEGtVBsqeEm,3375498602,52742162 to: AzH31mCMGTWGFKvKmsVe8Y51yEjkSqj5QomRRvvJ3SdE,52742163
[2020-12-15T01:14:53.682170379Z INFO solana_core::replay_stage] 71bhKKL89U3dNHzuZVZ7KarqV6XtHEgjXjvJTsguD11B reset PoH to tick 3375498496 (within slot 52742163). My next leader slot is 52742164
[2020-12-15T01:14:53.682195347Z INFO solana_core::replay_stage] new fork:52742164 parent:52742163 (leader) root:52742124
[2020-12-15T01:14:53.682717507Z INFO solana_metrics::metrics] datapoint: replay_stage-new_leader slot=52742164i leader="71bhKKL89U3dNHzuZVZ7KarqV6XtHEgjXjvJTsguD11B"
[2020-12-15T01:14:53.682731962Z INFO solana_metrics::metrics] datapoint: replay_stage-my_leader_slot slot=52742164i
[2020-12-15T01:14:53.686186552Z INFO solana_core::replay_stage] LEADER CHANGE at slot: 52742164 leader: 71bhKKL89U3dNHzuZVZ7KarqV6XtHEgjXjvJTsguD11B. I am now the leader
[2020-12-15T01:14:53.687682631Z INFO solana_metrics::metrics] datapoint: bank-new_from_parent-heights slot_height=52742164i block_height=42225190i
[2020-12-15T01:14:53.687707238Z INFO solana_metrics::metrics] datapoint: bank-timestamp get_timestamp_estimate_us=461i
[2020-12-15T01:14:53.687716570Z INFO solana_metrics::metrics] datapoint: bank-timestamp-correction slot=52742164i from_genesis=1602191932i corrected=1602191932i ancestor_timestamp=1602191931i
[2020-12-15T01:14:53.687722049Z INFO solana_metrics::metrics] datapoint: replay-loop-timing-stats total_elapsed_us=1079000i compute_bank_stats_elapsed=46804i select_vote_and_reset_forks_elapsed=26i start_leader_elapsed=11954i reset_bank_elapsed=335i voting_elapsed=24762i select_forks_elapsed=13i compute_slot_stats_elapsed=70i generate_new_bank_forks_elapsed=17073i replay_active_banks_elapsed=170787i reset_duplicate_slots_elapsed=6115i
[2020-12-15T01:14:53.687731208Z INFO solana_metrics::metrics] datapoint: broadcast-process-shreds-stats slot=52742165i shredding_time=20935i receive_time=68290i num_data_shreds=168i slot_broadcast_time=3i
[2020-12-15T01:14:53.687736425Z INFO solana_metrics::metrics] datapoint: block-commitment-cache aggregate-commitment-ms=5i highest-confirmed-root=52742122i highest-confirmed-slot=52742162i
[2020-12-15T01:14:53.687744008Z INFO solana_metrics::metrics] datapoint: shred_insert_is_full total_time_ms=159i slot=52742165i last_index=168i
[2020-12-15T01:14:53.687748891Z INFO solana_metrics::metrics] datapoint: broadcast-insert-shreds-stats slot=52742165i end_to_end_elapsed=153836i insert_shreds_elapsed=20456i num_shreds=337i
[2020-12-15T01:14:53.694297011Z ERROR solana_metrics::counter] COUNTER:{"name": "bank-process_transactions-error_count", "counts": 39312254, "samples": 39251000, "now": 1607994893694, "events": 1}
[2020-12-15T01:14:53.697277751Z INFO solana_metrics::counter] COUNTER:{"name": "bank-process_transactions", "counts": 239983035, "samples": 226285000, "now": 1607994893697, "events": 1}
[2020-12-15T01:14:53.704202396Z INFO solana_metrics::counter] COUNTER:{"name": "banking_stage-dropped_tx_before_forwarding", "counts": 1416, "samples": 2910000, "now": 1607994893704, "events": 0}
[2020-12-15T01:14:53.721436977Z INFO solana_metrics::metrics] datapoint: broadcast-transmit-shreds-stats slot=52742165i end_to_end_elapsed=187568i transmit_elapsed=16756i send_mmsg_elapsed=5696i get_peers_elapsed=539611i num_shreds=337i shred_select=4864i
[2020-12-15T01:14:53.797081131Z INFO solana_metrics::counter] COUNTER:{"name": "vote-native", "counts": 238078001, "samples": 238078000, "now": 1607994893797, "events": 1}
[2020-12-15T01:14:53.830913651Z ERROR solana_metrics::counter] COUNTER:{"name": "bank-process_transactions-error-duplicate_signature", "counts": 1268301, "samples": 1267000, "now": 1607994893830, "events": 1}
[2020-12-15T01:14:53.834868606Z INFO solana_metrics::counter] COUNTER:{"name": "bank-process_transactions", "counts": 239984046, "samples": 226286000, "now": 1607994893834, "events": 1}
[2020-12-15T01:14:53.835883655Z ERROR solana_metrics::counter] COUNTER:{"name": "bank-process_transactions-error_count", "counts": 39313254, "samples": 39252000, "now": 1607994893835, "events": 1}
[2020-12-15T01:14:53.841676150Z INFO solana_metrics::counter] COUNTER:{"name": "banking_stage-dropped_tx_before_forwarding", "counts": 1416, "samples": 2911000, "now": 1607994893841, "events": 0}
[2020-12-15T01:14:53.857925170Z INFO solana_metrics::counter] COUNTER:{"name": "bank-process_transactions", "counts": 239985046, "samples": 226287000, "now": 1607994893857, "events": 1}
[2020-12-15T01:14:53.860077648Z ERROR solana_metrics::counter] COUNTER:{"name": "bank-process_transactions-error_count", "counts": 39314254, "samples": 39253000, "now": 1607994893860, "events": 1}
[2020-12-15T01:14:53.863618027Z ERROR solana_metrics::counter] COUNTER:{"name": "bank-process_transactions-error-duplicate_signature", "counts": 1269301, "samples": 1268000, "now": 1607994893863, "events": 1}
[2020-12-15T01:14:53.864307425Z INFO solana_metrics::counter] COUNTER:{"name": "banking_stage-dropped_tx_before_forwarding", "counts": 1417, "samples": 2912000, "now": 1607994893864, "events": 0}
[2020-12-15T01:14:53.900624376Z INFO solana_metrics::counter] COUNTER:{"name": "broadcast_service-entries_received", "counts": 1252806, "samples": 597000, "now": 1607994893900, "events": 1}
[2020-12-15T01:14:53.914989943Z INFO solana_metrics::counter] COUNTER:{"name": "poh_recorder-tick_lock_contention", "counts": 30850623, "samples": 57828000, "now": 1607994893914, "events": 1}
[2020-12-15T01:14:53.915020116Z INFO solana_metrics::counter] COUNTER:{"name": "poh_recorder-tick_overhead", "counts": 10362763, "samples": 57828000, "now": 1607994893915, "events": 4}
[2020-12-15T01:14:53.926144607Z INFO solana_metrics::metrics] datapoint: optimistic_slot_elapsed average_elapsed_ms=718i
[2020-12-15T01:14:53.926166703Z INFO solana_metrics::metrics] datapoint: optimistic_slot slot=52742163i
[2020-12-15T01:14:54.018719265Z INFO solana_core::poh_recorder] poh_record: max_tick_height 3375498560 reached, clearing working_bank 52742164
[2020-12-15T01:14:54.019165893Z INFO solana_metrics::metrics] datapoint: shred_insert_is_full total_time_ms=338i slot=52742164i last_index=281i
[2020-12-15T01:14:54.019389620Z INFO solana_metrics::metrics] datapoint: broadcast-process-shreds-stats slot=52742164i shredding_time=36557i receive_time=167815i num_data_shreds=282i slot_broadcast_time=332129i
[2020-12-15T01:14:54.019397976Z INFO solana_metrics::metrics] datapoint: broadcast-insert-shreds-stats slot=52742164i end_to_end_elapsed=332263i insert_shreds_elapsed=49727i num_shreds=564i
[2020-12-15T01:14:54.019422399Z INFO solana_core::replay_stage] bank frozen: 52742164
[2020-12-15T01:14:54.019428408Z INFO solana_metrics::metrics] datapoint: replay-slot-stats slot=52742164i fetch_entries_time=0i fetch_entries_fail_time=0i entry_poh_verification_time=0i entry_transaction_verification_time=0i replay_time=0i replay_total_elapsed=332523i total_entries=0i total_shreds=0i
[2020-12-15T01:14:54.019477415Z INFO solana_metrics::metrics] datapoint: collect_rent_eagerly accounts=2i
[2020-12-15T01:14:54.020297104Z INFO solana_metrics::metrics] datapoint: broadcast-transmit-shreds-stats slot=52742164i end_to_end_elapsed=333155i transmit_elapsed=30889i send_mmsg_elapsed=10428i get_peers_elapsed=1128903i num_shreds=564i shred_select=7559i
[2020-12-15T01:14:54.026951753Z INFO solana_runtime::bank] bank frozen: 52742164 hash: BEzVL1jAQ6hx7btWQpsynkeShqokvER6NFHLU6rYbU2e accounts_delta: 2wAmuKjrDBBmG2ZtzN4W2Vug8DE7mvTtG1cpAiHykLrX signature_count: 378 last_blockhash: Dxg1VvUcS3rb4hNQQF3ye1JPpRsf634Zy85yHhZZq4Pk capitalization: 1293979802488999998
[2020-12-15T01:14:54.027016140Z INFO solana_runtime::bank] accounts hash slot: 52742164 stats: BankHashStats { num_updated_accounts: 1033, num_removed_accounts: 0, num_lamports_stored: 620107668240618210, total_data_len: 1444836, num_executable_accounts: 0 }
[2020-12-15T01:14:54.034357857Z INFO solana_metrics::metrics] datapoint: tower-observed slot=52742163i root=52742123i
[2020-12-15T01:14:54.037810078Z INFO solana_core::replay_stage] 71bhKKL89U3dNHzuZVZ7KarqV6XtHEgjXjvJTsguD11B slot_weight: 52742164 0 0 52742163
[2020-12-15T01:14:54.037818768Z INFO solana_metrics::metrics] datapoint: best_slot slot=52742164i best_slot=52742164i
[2020-12-15T01:14:54.037847787Z INFO solana_metrics::metrics] datapoint: bank_weight slot=52742164i weight="0"
[2020-12-15T01:14:54.037850959Z INFO solana_core::replay_stage] voting: 52742164 0
[2020-12-15T01:14:54.037879601Z INFO solana_metrics::metrics] datapoint: tower-vote latest=52742164i root=52742125i
[2020-12-15T01:14:54.050049625Z INFO solana_metrics::metrics] datapoint: purge_slots_time remove_storages_elapsed=0i drop_storage_entries_elapsed=0i num_slots_removed=0i total_removed_storage_entries=0i total_removed_bytes=0i
[2020-12-15T01:14:54.051353093Z INFO solana_core::replay_stage] new root 52742125
[2020-12-15T01:14:54.051532464Z INFO solana_core::replay_stage] vote bank: Some((52742164, SameFork)) reset bank: 52742164
[2020-12-15T01:14:54.051558165Z INFO solana_metrics::metrics] datapoint: blocks_produced num_blocks_on_fork=534090i num_dropped_blocks_on_fork=110205i
[2020-12-15T01:14:54.051610063Z INFO solana_core::poh_recorder] reset poh from: 7XcLJ5N1ZMCZMfkBmkxYDJq4QA4HYeByCTgEXE4dMKVX,3375498566,52742164 to: Dxg1VvUcS3rb4hNQQF3ye1JPpRsf634Zy85yHhZZq4Pk,52742164
[2020-12-15T01:14:54.051644619Z INFO solana_core::replay_stage] 71bhKKL89U3dNHzuZVZ7KarqV6XtHEgjXjvJTsguD11B reset PoH to tick 3375498560 (within slot 52742164). My next leader slot is 52742166
[2020-12-15T01:14:54.054936840Z INFO solana_metrics::metrics] datapoint: block-commitment-cache aggregate-commitment-ms=3i highest-confirmed-root=52742123i highest-confirmed-slot=52742162i
[2020-12-15T01:14:54.084916407Z INFO solana_metrics::metrics] datapoint: poh-service ticks=128i hashes=1613921i elapsed_ms=661i
[2020-12-15T01:14:54.309099253Z INFO solana_metrics::counter] COUNTER:{"name": "banking_stage-buffered_packets", "counts": 894685, "samples": 847000, "now": 1607994894309, "events": 1}
[2020-12-15T01:14:54.356949022Z INFO solana_metrics::metrics] datapoint: replay_stage-new_leader slot=52742166i leader="71bhKKL89U3dNHzuZVZ7KarqV6XtHEgjXjvJTsguD11B"
[2020-12-15T01:14:54.356995304Z INFO solana_metrics::metrics] datapoint: replay_stage-skip_leader_slot slot=52742166i parent_slot=52742164i latest_unconfirmed_leader_slot=52742164i
[2020-12-15T01:14:54.356937487Z INFO solana_core::progress_map] Propagated stats:
total staked: 12589672443775499,
observed staked: 164526072309456,
vote pubkeys: {wn7GxfuY8GGncX5DPuzkuQ5UXhoRdcq8goiYU1Y2e5w, ARxbupmvHEsttmDye1iafygCSMFAoAcEDTdVUWy8ZMjD, BSxPcwvsMszsPr7h9kufpSqnhiyFyzCVAgSDC8r2A7P8, Fy4bjUvtDVtuy7HhmEYX2Sy7BeeQVhriKMbSkevt2KQd, 8zipsAVJU28GyirnyUNwt2yjTNuNusq3ZiJoVn41EgJE, 4bT7T61czU7mZEZzbUorH3YwS429SijxMhQLungEb2Dw, BpP3GAPvs9rR7iieLyzhPvJwaYYHuckhzC9xBmhb9yz, ANNVFjexVL12ugyL1KS5bbEr7NzPx1VkZk4XroCgYyws, 7eSDCDd2J2GHjp8DxyxihqBrSLY7Sapq3S9t4EiXz2Lh},
node_pubkeys: {},
slot: 52742164,
epoch: Some(134)
[2020-12-15T01:14:54.357119168Z INFO solana_metrics::metrics] datapoint: replay_stage-retransmit slot=52742164i
[2020-12-15T01:14:54.365723984Z INFO solana_metrics::metrics] datapoint: shred_fetch_tvu_forwards index_overrun=0i shred_count=166i slot_bad_deserialize=0i index_bad_deserialize=0i index_out_of_bounds=0i slot_out_of_range=0i duplicate_shred=37i
[2020-12-15T01:14:54.365979170Z INFO solana_metrics::metrics] datapoint: shred_fetch index_overrun=0i shred_count=3i slot_bad_deserialize=0i index_bad_deserialize=0i index_out_of_bounds=0i slot_out_of_range=0i duplicate_shred=0i
[2020-12-15T01:14:54.457630279Z INFO solana_core::replay_stage] new fork:52742168 parent:52742163 root:52742125
[2020-12-15T01:14:54.458996749Z INFO solana_metrics::metrics] datapoint: bank-new_from_parent-heights slot_height=52742168i block_height=42225190i
[2020-12-15T01:14:54.461724085Z INFO solana_metrics::metrics] datapoint: bank-timestamp get_timestamp_estimate_us=399i
[2020-12-15T01:14:54.461743599Z INFO solana_metrics::metrics] datapoint: bank-timestamp-correction slot=52742168i from_genesis=1602191934i corrected=1602191934i ancestor_timestamp=1602191931i
[2020-12-15T01:14:54.488282179Z INFO solana_metrics::metrics] datapoint: replay_stage-new_leader slot=52742166i leader="71bhKKL89U3dNHzuZVZ7KarqV6XtHEgjXjvJTsguD11B"
[2020-12-15T01:14:54.585423866Z INFO solana_metrics::metrics] datapoint: shred_fetch_repair index_overrun=0i shred_count=5i slot_bad_deserialize=0i index_bad_deserialize=0i index_out_of_bounds=0i slot_out_of_range=0i duplicate_shred=0i
[2020-12-15T01:14:54.589999444Z INFO solana_metrics::metrics] datapoint: replay_stage-new_leader slot=52742166i leader="71bhKKL89U3dNHzuZVZ7KarqV6XtHEgjXjvJTsguD11B"
[2020-12-15T01:14:54.611896539Z INFO solana_metrics::metrics] datapoint: cluster_slots_service-timing lowest_slot_elapsed=242496i update_completed_slots_elapsed=138305i
[2020-12-15T01:14:54.636949040Z INFO solana_core::retransmit_stage] retransmit: packets_by_slot: {52742162: 7, 52742163: 677, 52742168: 546}
[2020-12-15T01:14:54.636957312Z INFO solana_metrics::metrics] datapoint: retransmit-num_nodes count=0i
[2020-12-15T01:14:54.637010922Z INFO solana_metrics::metrics] datapoint: retransmit-stage total_time=127434i epoch_fetch=2242i epoch_cache_update=10571i total_batches=865i total_packets=1234i retransmit_total=22125i compute_turbine=77263i repair_total=12i discard_total=249i
[2020-12-15T01:14:54.636985007Z INFO solana_core::retransmit_stage] retransmit: packets_by_source: {"103.218.240.68:17005": 1, "116.202.109.229:8005": 1, "116.202.174.169:8006": 7, "116.202.192.253:7617": 5, "116.202.195.253:8006": 4, "116.202.226.51:8005": 2, "116.202.232.138:18005": 3, "116.202.232.85:18005": 3, "116.202.233.120:8006": 8, "116.202.234.103:8006": 5, "116.202.236.82:8006": 3, "116.202.237.5:8006": 4, "116.203.110.213:8005": 4, "116.203.192.255:8006": 1, "134.122.75.20:9056": 8, "135.181.112.92:11005": 2, "135.181.113.110:8005": 4, "135.181.114.157:8005": 4, "135.181.114.15:8005": 4, "135.181.116.161:8005": 1, "135.181.116.223:8005": 2, "135.181.116.99:8005": 2, "135.181.117.109:8005": 1, "135.181.119.35:8005": 3, "135.181.119.99:8005": 2, "135.181.129.188:8005": 1, "135.181.130.166:8007": 3, "135.181.132.189:8005": 5, "135.181.133.242:8005": 1, "135.181.134.158:8005": 2, "135.181.134.159:8005": 2, "135.181.136.94:9006": 3, "135.181.137.91:6005": 2, "135.181.138.113:8007": 2, "135.181.139.218:8205": 1, "135.181.140.117:8006": 8, "135.181.140.118:8006": 3, "135.181.16.163:8005": 5, "135.181.16.248:8006": 4, "135.181.17.13:10006": 1, "135.181.17.60:9006": 1, "135.181.18.150:8006": 2, "135.181.19.233:9006": 1, "135.181.19.71:10006": 2, "135.181.19.95:8006": 3, "135.181.2.116:8006": 2, "135.181.2.18:8006": 1, "135.181.20.212:8005": 9, "135.181.20.213:8005": 7, "135.181.20.214:8005": 6, "135.181.20.218:8005": 1, "135.181.20.254:8005": 7, "135.181.21.46:8006": 4, "135.181.3.34:8005": 2, "135.181.3.50:8005": 1, "135.181.3.95:8005": 3, "135.181.4.102:8005": 4, "135.181.4.115:8006": 5, "135.181.4.29:8205": 2, "135.181.5.112:8005": 6, "135.181.5.147:6006": 1, "135.181.5.151:8006": 5, "135.181.5.152:8006": 1, "135.181.5.164:8005": 3, "135.181.56.146:8005": 5, "135.181.56.35:8005": 4, "135.181.57.169:8005": 2, "135.181.57.170:8005": 2, "135.181.57.31:8005": 4, "135.181.58.47:8005": 3, "135.181.59.214:8005": 3, "135.181.59.242:8005": 1, "135.181.6.112:8005": 9, "135.181.6.223:8005": 6, "135.181.60.146:8006": 3, "135.181.60.248:8005": 1, "135.181.61.190:8007": 7, "135.181.63.116:8005": 2, "135.181.7.30:8006": 5, "135.181.73.109:28006": 6, "135.181.73.157:8005": 4, "135.181.73.16:8005": 3, "135.181.73.32:8005": 2, "135.181.73.99:8005": 5, "135.181.74.243:8005": 1, "135.181.74.33:8005": 7, "135.181.75.231:8005": 3, "135.181.75.94:8005": 1, "135.181.76.119:8005": 5, "135.181.76.121:8005": 5, "135.181.76.174:8008": 1, "135.181.76.49:8106": 1, "135.181.76.52:8005": 3, "135.181.76.93:8005": 2, "135.181.76.95:8005": 4, "135.181.76.96:8005": 5, "135.181.76.97:8005": 2, "135.181.77.179:8005": 2, "135.181.77.214:8005": 6, "135.181.77.225:8005": 5, "135.181.79.174:8005": 3, "136.243.134.221:8006": 3, "136.243.138.236:8006": 5, "136.243.146.94:8005": 2, "136.243.153.94:8005": 2, "136.243.174.152:8006": 6, "136.243.65.35:8006": 6, "138.201.135.79:8006": 6, "138.201.207.179:8006": 4, "138.201.52.211:8205": 2, "138.201.85.2:8005": 2, "138.201.96.141:8006": 8, "144.76.143.68:8006": 4, "144.91.125.237:8005": 6, "148.251.19.197:8005": 1, "150.238.34.227:8005": 4, "159.69.157.138:8005": 4, "159.69.58.173:8006": 2, "159.69.59.60:8006": 6, "161.97.68.14:8005": 6, "161.97.68.65:8005": 4, "161.97.71.26:8005": 1, "167.86.119.206:8005": 4, "167.86.121.139:8005": 4, "167.86.86.150:8005": 2, "167.86.99.50:8005": 6, "168.119.115.16:8005": 10, "168.119.123.78:8005": 7, "168.119.137.101:8006": 8, "168.119.141.61:8006": 1, "168.119.149.185:8006": 4, "168.119.164.61:8005": 4, "168.119.166.8:8005": 3, "168.119.182.61:8005": 6, "168.119.38.49:8005": 2, "168.119.4.39:8006": 4, "168.119.4.40:8006": 3, "168.119.49.27:8006": 4, "168.119.60.248:8005": 8, "168.119.64.90:8006": 3, "168.119.66.228:8006": 1, "168.119.73.184:8006": 6, "173.212.218.178:8005": 3, "173.212.239.95:8005": 1, "173.212.248.212:8005": 2, "173.249.58.103:8005": 3, "176.9.152.171:8006": 1, "178.162.153.70:8006": 6, "178.170.40.112:8005": 2, "178.170.40.38:8006": 2, "178.63.101.35:8006": 1, "178.63.97.232:8006": 1, "185.144.83.136:8006": 7, "185.144.83.146:8006": 2, "188.191.36.218:8005": 3, "188.191.36.223:8005": 2, "188.40.122.93:8006": 2, "188.40.132.97:8006": 2, "188.40.141.235:8006": 3, "195.191.219.107:8005": 2, "195.201.138.38:8006": 2, "195.201.160.171:8006": 5, "195.201.162.228:8005": 4, "195.201.59.122:8005": 1, "206.189.26.38:8005": 1, "207.180.194.166:8005": 2, "207.180.255.2:8005": 3, "209.222.97.92:8005": 2, "213.239.219.35:8905": 4, "213.246.56.247:8006": 1, "3.128.29.187:8005": 3, "35.227.127.245:11005": 5, "46.4.67.225:8006": 1, "49.12.0.172:8005": 8, "49.12.132.118:8005": 6, "49.12.132.30:8006": 3, "5.9.17.92:8006": 5, "51.105.24.2:8005": 7, "51.89.194.50:8005": 3, "54.162.38.181:8005": 3, "62.171.170.50:8005": 1, "62.171.174.135:8005": 3, "62.251.42.161:10006": 12, "78.154.165.67:8005": 6, "78.154.165.68:8005": 6, "78.154.165.69:8005": 4, "78.46.101.181:8005": 9, "78.46.103.148:8006": 4, "78.46.124.140:8005": 1, "78.46.217.155:8005": 3, "78.46.41.60:8006": 7, "78.46.83.115:8006": 1, "78.46.87.157:8006": 2, "78.47.100.250:8005": 4, "78.47.127.50:8005": 6, "80.224.76.172:7617": 23, "80.224.76.172:7620": 3, "85.190.150.169:8005": 76, "88.198.24.93:8005": 4, "88.198.37.45:8005": 7, "88.99.168.180:8005": 3, "88.99.186.187:8005": 2, "88.99.58.143:8005": 3, "88.99.6.218:8006": 2, "89.201.4.237:8005": 7, "89.42.234.16:8005": 3, "89.42.234.18:8006": 2, "91.193.116.35:8005": 2, "91.90.148.195:8006": 3, "92.253.253.55:8005": 4, "92.253.253.60:8005": 2, "94.130.112.189:8006": 2, "94.130.183.15:8006": 5, "94.130.22.78:8005": 1, "94.130.53.140:8005": 3, "94.130.54.247:8006": 5, "95.216.10.105:8205": 1, "95.216.10.148:8005": 4, "95.216.10.172:8006": 2, "95.216.12.141:8005": 4, "95.216.158.252:8005": 9, "95.216.168.143:8006": 8, "95.216.17.58:8006": 3, "95.216.18.178:8006": 7, "95.216.18.224:18005": 2, "95.216.18.57:8005": 4, "95.216.224.228:8005": 3, "95.216.225.31:8005": 1, "95.216.226.59:8005": 3, "95.216.24.146:8005": 2, "95.216.24.251:8005": 6, "95.216.24.66:8005": 2, "95.216.247.189:8005": 1, "95.216.29.87:8005": 2, "95.216.37.122:8006": 1, "95.216.41.41:8005": 5, "95.216.41.57:8005": 6, "95.216.42.41:8005": 2, "95.216.46.162:8005": 1, "95.216.67.98:8006": 3, "95.216.7.166:8006": 1, "95.216.76.163:8005": 7, "95.216.76.181:8005": 3, "95.216.77.107:8005": 2, "95.216.78.108:8005": 2, "95.216.8.228:8005": 3, "95.216.9.55:8005": 1, "95.216.96.176:8005": 3, "95.216.97.45:8005": 1, "95.217.114.150:8006": 2, "95.217.114.32:8205": 1, "95.217.118.253:8006": 9, "95.217.118.36:18005": 1, "95.217.118.37:10005": 2, "95.217.119.101:8305": 11, "95.217.119.108:18005": 5, "95.217.119.123:8806": 4, "95.217.121.146:8005": 6, "95.217.121.152:8006": 6, "95.217.121.252:18005": 7, "95.217.144.25:18005": 1, "95.217.145.161:8005": 7, "95.217.146.58:8006": 9, "95.217.179.195:8006": 9, "95.217.181.253:8005": 4, "95.217.192.210:8006": 3, "95.217.192.222:8005": 8, "95.217.193.110:8006": 1, "95.217.193.34:8005": 5, "95.217.193.36:8005": 8, "95.217.193.43:18005": 3, "95.217.195.236:8006": 2, "95.217.195.236:9006": 5, "95.217.196.110:8005": 5, "95.217.196.126:8005": 13, "95.217.196.224:8006": 4, "95.217.197.117:8005": 3, "95.217.203.114:8005": 5, "95.217.203.156:8005": 3, "95.217.203.62:8006": 4, "95.217.204.22:8005": 7, "95.217.204.230:8205": 2, "95.217.206.164:8005": 2, "95.217.206.164:9006": 4, "95.217.207.151:8005": 5, "95.217.207.162:8005": 2, "95.217.207.169:8005": 4, "95.217.207.215:8005": 3, "95.217.207.55:8006": 8, "95.217.209.217:8006": 4, "95.217.211.131:8006": 4, "95.217.224.95:8205": 2, "95.217.225.58:8005": 1, "95.217.226.116:8013": 1, "95.217.226.222:8006": 4, "95.217.227.217:8005": 8, "95.217.227.245:8205": 4, "95.217.230.217:8005": 6, "95.217.230.222:8005": 1, "95.217.230.26:8005": 4, "95.217.231.23:9006": 1, "95.217.231.39:10006": 1, "95.217.231.54:8005": 2, "95.217.3.69:8005": 6, "95.217.36.182:8006": 2, "95.217.39.26:8005": 3, "95.217.41.120:8005": 4, "95.217.62.175:8005": 4, "95.217.74.231:8006": 1, "95.217.78.33:8006": 6, "95.217.82.61:9006": 1, "95.217.82.62:9006": 1, "95.217.83.214:8005": 1, "95.217.98.53:18005": 2}
[2020-12-15T01:14:54.651607525Z INFO solana_metrics::metrics] datapoint: do_shrink_slot_time storage_read_elapsed=1i index_read_elapsed=0i find_alive_elapsed=0i create_and_insert_store_elapsed=0i store_accounts_elapsed=0i update_index_elapsed=0i handle_reclaims_elapsed=0i write_storage_elapsed=0i rewrite_elapsed=0i drop_storage_entries_elapsed=0i
[2020-12-15T01:14:54.709707686Z INFO solana_metrics::metrics] datapoint: replay_stage-new_leader slot=52742167i leader="71bhKKL89U3dNHzuZVZ7KarqV6XtHEgjXjvJTsguD11B"
[2020-12-15T01:14:54.709731332Z INFO solana_metrics::metrics] datapoint: replay_stage-skip_leader_slot slot=52742167i parent_slot=52742164i latest_unconfirmed_leader_slot=52742164i
[2020-12-15T01:14:54.709696547Z INFO solana_core::progress_map] Propagated stats:
total staked: 12589672443775499,
observed staked: 2867704371454897,
vote pubkeys: {Bj9voGPwxCe9SWG1BAU8ddzfsFUkuYgGq56Gy2eE5H74, GpLc3JsjHWkjFqDdGGS1oyasaYutHS8ui87E2vrrNqjH, 123JiW1rwJ4JM5BxYqFkBi6wQJ52pt6qUH88JDqrtU9i, GYJoPabxvQUgWrgKphTEFCzAKSXaSxgd34vsvV2VAWV2, 2CqUJ8sxy1CUomZMnvVaDgb4yvt61z8o1wfsJwES4Ct2, 4HdZFdBQTKhXUq9tRACuFVBTwh8F1973ikSksrxzkvMH, 6vLvQwNZY6GZ5LgoARyn7ZVTXanKo7Ng1m9FRLE6P8uB, AT1kLQm1uYB5XZVocNUn9QkH3NPJRjRvf5orGS469xuy, BSxPcwvsMszsPr7h9kufpSqnhiyFyzCVAgSDC8r2A7P8, DhKKfZ2gzpbGfB5z5zmJjD2Q6rsm2UHTRx5aHVtx2PSa, 7qC71AhdVQ45vMvbJc7crmQ46CMf2Pb1RkZd1cxHqNce, HGva6MRM6QinEidGv1oAQsvyLziHsnG8EkTPiaEDBRNL, B78hrhYa8uXpJx9xUQaHic7M5R1Wrz7JWQzSMD7t6nt4, 8VpgwtBxMWhCiaXHXabgfsEQvhV8CrndbxZSfNuZELSx, 2WBWd6AhVitsCoaqcGYKjZcXUhj8oCMA7PrZbpzZd4Ke, XVeGiHR2gaNBp1tP6NMAyEEQC7JcTFw5uTQyJRPgLME, 5VkqsPyR6zD8eb2P3j2PwJZwQFekKgAGA3hUtUYr1ra, CY2GoofFsrvk9uF2QFcbiEzJnzRbtcsgAJ8J8UjecMiW, 9t6AMT1sq3LuASWqb21qduMMwVMbCDujKLfSQwxre2zK, 66jaSW12UXQ3iyAuCWT1ybPuKt27m1D4Yv4NTBFDRTc6, CVHqXZYPggbU289HhEVKzcCBW5wjVe4F2Msbg7gzegUo, 59TjvREZcxyBmXkxvcc3kCSeCPn6WAF5Mjvkty2pCiMp, FHrN2p9JzC8kiHee6rzddT4QokbXAxFqb8HxBcP5uY96, E9XgpPUhcfDRuHhdiC6Hsr4cJMzfcM7xAeSiKw2caD5G, 6AWNt6g5RPePMiDVDj3NV5JVs3m8tt5kLkaHbtssnfTm, 7CXY5rrdLGDVCfMiRqfXuA5GFsRx8CGYGPXSJ1tdcday, 3vU3mYXjRA1uKpRqETAju4shLLgBSzgCSyhoJYw5sNWy, 59G1GgyGV3hzdqphsZnEfa3VDdd1c8g96gqwgEgBZxRB, 7cwT5hFDTkVGgc15nmriizgwXHr7BX6TJPs5ztoDThsi, AU9jBbFA6b5Hd8xGTcK7HYkvvT5dHcYry8KLqzdcE7cP, 69rUCD1qk6hJ3AqEkCi236QaceHTBSv1isiCEW6EZnVa, BF4trJ3vr7J2MwSJkDqRQYqG4QHJXpCoiFhzsSg8sSBF, A6mk9xHgqp96W6ytM9xiKEWwji6goak9imBJyQEHNDeq, ARxbupmvHEsttmDye1iafygCSMFAoAcEDTdVUWy8ZMjD, GPUCxZegum4LtjcWTzubq97H3LPScSzDJyCmRxiNKxQi, EzAs3K3eWn9WeWWcp1LbPGyPcPRD1euJh7teu2gtRrNn, 5XQ1w7hj7kRGuzPbwTZ6cVEB8PBQdDkeiaDHmWPpoehp, Fy4bjUvtDVtuy7HhmEYX2Sy7BeeQVhriKMbSkevt2KQd, 8zipsAVJU28GyirnyUNwt2yjTNuNusq3ZiJoVn41EgJE, 4i2FSQfm4dKuvJEVREo4TwbbrF1srcRByEfb8mfgU29x, 6rhtc9z8n7QK5gtJTurdxAfJwJStrQJjU4vaSEbCANdX, HvpziFn6Wov3FU9zxg6qMahb1ufzA31J4985LBL5amcN, GQmHP4knX718u2AgMZkv6UARxjasgzN52NarBK5hBZLg, DcN1k7WD6BH7ez3aNLKn4a3inK8Bkz4dP562gs91aMhc, 7Y8A18kf996yCesXp7APt6Lt8zrXUVvcHiVLZxPKbV5L, 7NaeGmcq8wvz3EnLkP5n6thmnKyza8nJaua4kcpbfY1n, 8JgBYBuYoQnSwz1zhGuKJx7agsGoNXDeRmHdKZkMMLQY, 9JDRX41qKtqmkRovDHAiY4r7vqSXsm5nXyhuc2C5Aowy, J6tZDkc3YqRdxLacfNrVLA117co268kED38ni87wxQh9, 6PrdarUjqPbW4cgY6ehFnLJdhbWKCFv728tnu4itPv7E, Fp6Uek5WAjdDDfUpuKWeG8pvdh2KFLLnK9Es3JUHaZXv, 5EiXqUq8gv5RSknKauUTpF2c7oR3L7fSZkD55eJC16UL, BBgnRoNkd1poQLfJ5x7yVs1SRfo8RykdjvYHXQ3NvKzC, 8pKN3VU3qkYVPX9QJeijGoTQiTtmvu9QDvQAXfuXfv2s, G9ZfVCWk1XFEaju4LwpQNcUHGBNCFZ9JU6L1q6qyY7W2, 2zY1fMcEn9wiUjdMHzz2ZKokJ3oWFKhr4CX6ewznyJtB, 4bT7T61czU7mZEZzbUorH3YwS429SijxMhQLungEb2Dw, 7eU2Ryqh1MnGCdLw6kdR7BhsCSoSRGu5AHf674s2Mj3J, 6H8zfzpZ4jEg4DhwaHvdjgHMWfB2DFmofNngSgSpJzn8, sohoFABjRFFAdoqDTqtwhsBzJeNUHtATho552aa5zCw, 3MdPz8QZzggSMViQWkrbS7nMhuGhb52vkBaPLnCmmqBt, 7eSDCDd2J2GHjp8DxyxihqBrSLY7Sapq3S9t4EiXz2Lh, H1BQ8C8u58tAUvMaR2GimuYmYQaeGBmAajVxW4i4L4DE, 8bChvXwAci1TP2F3KmeghESgMY4LYJicpx3zpnQj1Z8n, 3LRfXRmRiRXqAZW8fnJK8DFF6gUFWzcvFY6FBepyVPqC, 7PCfYyD9ZGuERWqMFd4YWgfi86vcvQ9W8b5Hb5Ta8BB6, YH7G4b5CWqhMvXP6ZDi49ZbTwkgkmEeLirpccmpNJZQ, 2FRNLV1V5VeqZLDksR5mbEJAiGFoZdyQGH94Ro4CnGAF, HApadLChwYdaEy5N1QDZsPLN1372naJfDmBbdV1WJWX6, 8CLBZQbMZvqo9oTwBkqrUeungHswhq65ZFmxLU3qHc7f, 38QX3p44u4rrdAYvTh2Piq7LvfVps9mcLV9nnmUmK28x, ANNVFjexVL12ugyL1KS5bbEr7NzPx1VkZk4XroCgYyws, CuAB9aMXrYu1Ey1dNfZ5caGQGiqbxenSaTJkWRst5PiV, FpK372qEPeiypkr6fGTzG8UzYEy7err4vpBwLaapavwC, EwQtkcJFQftsi8jak3k7K5p8iZHhDbHyMNVRrGwyksiW, 4MbS3kVBG39Q4pHe8BCsguB9KnbJPrGgvCdiQwSzRupU, 9uYpoD4xPXuUAj2HTJii9ngT3ATnkhNEy5qKn55J5j9h, BrD2kqHJjUKUsgiofuJiiXaxA94mZv49HBy6BbfCAvbj, Cefw1iG85GcADAUZRUNN5X2KMKcdY3hTrVzLAfABfso7, Deb9asvcCau6T2oxU7U3asEcYXMbGgUHYSnNx2XKSoEY, Bx7TJNxRWrATVEfXKWVWXhooVdUje1TUTEmWsgPtBUQn, EQejJRB5ELm4SwetQwysfHUg6EDVvCtzjUrtFrRPPu3F, 8UUgBAx8ekGZNjrNKCbpUZDZS8SC7iWWnnkrQvH4udfH, 6Po7XKFrGYVijoJdTZEGbCDAs6s9kS9LbN91Ew7rU4Hi, 4UixCSLvEraSCAJt1SztxkciKdFLMnddWAK3N9h91xge, 9f9APuGxhne1ek3QXu6f6xv6iNEdXjp4uimrGqvXcMQn, HNKfBiEqPsW9ehVjkP3FYQwMsfgG4CDYgtPLd9aKJVGy, Efjuc5sRR2LauiQjgquETZr24sHLbpM5QQV4jwAsFHDh, 56y3GnfskxYGEi1LsqnEyfFbUZLG8NwSZarZKje2qYfo, 7jEj5ydxhWj8WcrrkC4bJJhCVZtJc31LWLRXwVVfscyK, 7RQ5tqj2E98k7rk6jGNmpo2aMBLRSkqs2hgupC6yUeZP, T6S49frjfPEiSKpXYwSKmquKKACvbDadTFiT5zYs1z4, 2Tf1jSn18Wr9CeJsnMgnqbtHFJLuTSnVr1zqaM7SE62p, Hr1j3h4eKGLxzPtvZ6kdwVJxENRrTKR9qM94cMVLbsKM, 4WN4YE6b66ZYTPDJKxnVmsUuu1drGaNaYpxYL1ATyXJR, 2HUKQz7W2nXZSwrdX5RkfS2rLU4j1QZLjdGCHcoUKFh3, 8Ajgyy6NNnXX7uRYT8h8uxyBkW9XNhuq9TDJLTvUjz6N, BpP3GAPvs9rR7iieLyzhPvJwaYYHuckhzC9xBmhb9yz, BspiQEcRuVnv5xPmjs4EYBBSgGYeh9NpaTiqLSne6y1s, 5CSKsHA7JN5RozBZ2nDYCj1bNRgMvqsPtafyvdpgCJKC, HFZKQqNyqrMtxQfGLdFEziZo98FusornYjNc9q6HVaSL, 2c7dPJaGBW527993Jm5cGSdGxyJ4HYK56QGArLxkiddi, B5LXVkoov3duGbkp7EMLMMtQeB5JEToSWfMCC8iCfhnx, 2wRFSugXUnDsFU4JoWYtrhfdhX9L3Sai6xjjeYXaXtKy, 4fgGBXg9igcXQQWMNzW57WLA8r1DCmib6psxj3rea5jY, 81pUnzL2jirD2twTpjUeh2eT95YNaUtZHVQRi28Sbs1L, 5eWGkKXYAXZHpY1YkDACs8Hf9B9xcPYy7Vam5oKHWg66, 7zev81GZHxVrKeLxwLpg9byr4MBKtW8zf7MyybyTFmM4, wn7GxfuY8GGncX5DPuzkuQ5UXhoRdcq8goiYU1Y2e5w, 9b1Bs9dsJffEH4h1866w4ZjkVzpFMTgLkdvn4USXesVn, 87QuuzX6cCuWcKQUFZFm7vP9uJ72ayQD5nr6ycwWYWBG, 56XuFQKALiwAz9eMgpVKfs5kmKEEboeEXMiPWbrsPmiJ, 7kSG8Thuw6PUzbytQTUpR2nqAEK8xUVGB1yp266mdBYg, 8r1f8mwrUiYdg2Rx9sxTh4M3UAUcCBBrmRA3nxk3Z6Lm, 7FXvhWS2tpod5XkdMx31MbDo6Wv4xTH15rT8hcQ6Kx9E, ETDmLegMdJb9MH5ujaZNfWbGws8bfaZeokezqvpjYvi3, CAACr5HuSiftfVxCEXfUMCGZRbehUVj72wy2JnaonTky, CM7VKs1CHimuijv67wJV5BA4gCaf5HZAB6AWBRCaW2JT, J1cF3cFwBbUdoPjRx79VqpgEeReud8vByfkZEAvSNKWK, AQxWAUTJH4b7JEngTXTT1CDUAFWMvHveeFFbTswRHbTr, 1cgvkt3as5j9AkiiE6uaw6Bhhjz3X1wXxUr4ksiizWa, E2rqvVmCMUgDGYaXkMVifSfoMtfNJSBXKUnGjv38hmKF, EniTvDyRxjv6kvYTrBb6ySdZxH5Sns6hZPfsoF4jsMhX, 3B2x7waA9NkJY3wNaFXwp3ZYYs5HfasvZNFJRavU3q85, EydfUUyQ6xm2mJwQrvxWSV7YczYrF8yuvN2H2xqNycDc, haEUn8RDoSfkTmJKkagxfmVxaS7iZgeikKWp1ZzPLfd, Vty1mKNjEuwru1KYvDtLmzFTfK4R858jHfJD7kdUSUS, 3pC6V5gRjPpN2wbpYCkNqtXqyYK3jUbXdGsZeb2k6MeX, 6JzZtB7xBPSHLLzJLxZ6mPQm2Q3AcrDvT8nBVm4omwJQ, 8DS7rRQF5HDTZ8HmZjbD6U58t8KXfcpr395vajUi4pPu, Co8F1V2FHBntMHVHCPDGMxju8H74jEDeZXutGYdxaB51, 2w31fB2LiG3QsTKms6wwEyxcnnAT3dbDpDHMGSYXnAH9, JE41LoUK28Yfxje3hgTum7qTPPWomKfvuBPePzet9k1M, Cgs5uBf9zBbk2GHe1Rn82QXpz3azysUYHsiVkFc49AK3},
node_pubkeys: {8ZjS3d1bQihC3p5voM8by2xi5PoBNxzTJtaQ9rvxUbbB, BVLVnUm2tkgX1sK2f5oVtTMR1opGra72s4qBD9LjMd9Q, 3HitRjngqhAgVuNdFwtR1Lp5tQavbJri8MvKUq5Jpw1N, FVnG988wW3uF613QVxmQrkwtdzS8taxjFsuARTzZBwMo, GbtVg3D6bNFSjem21vrJBJTpUniwwEtmvs8mQkX5XS1V, 5DsrdX4xPok2YNHUEtQsRuyAkDcdSBPXM74ezfRgy8Vm, 3i7sS5McrJ7EzU8nbdA5rcXT9kNiSxLxhwyfuxbsDvBj, 5NLjk9HANo3C9kRfxu63h2vZUD1cER2LacWD7idoJtKF, 4Bx5bzjmPrU1g74AHfYpTMXvspBt8GnvZVQW3ba9z4Af, Hu7DW7BoXXuKbwaFJaAMEXpBv8pqBJPhfThMD96WHiJR, Hkj4Y4QxFvyoCd2wzAswsDpwW4vD1vyC5vppVyDDhJ8F, FbWq9mwUQRNVCAUdcECF5yhdwABmcnsZ6a6zpixeKuQE, 71bhKKL89U3dNHzuZVZ7KarqV6XtHEgjXjvJTsguD11B, HmSU5YJr4XK2SYdF6dxNXtF9PQRzbXXupUXCVEaJZX37, 123vij84ecQEKUvQ7gYMKxKwKF6PbYSzCzzURYA4xULY, Db2V7nPHc4sPHne87nYXPGn8Kv8rMsiWCAjgAXmpqcpC, HxnjZ5Qg59nupGGXVo77idUxfsiRXPcBbBt2hw3Nt99c, 2dm9YbgXtR5yimmgsLkfaMLcNZxhjywW4bLnvChms3tb, 7sEpbQB3Dryn5JhQVCGWoGgUfYwNEZzjPNa1Tu9mVa5p, Drf2oN83THfrUJHA9AGzJZaL9KMKggPoL9HJVttkSCgL, 2VzCLy98rzmvKGo23e1eM4LANCt9JFrtVTBBMZzGT4FW, 9xe4rcxYUe6iADdnvLkWn8K26bvyWgfrp9HYbtwR2sPs, J78SNwDW6G86sMmh7djnBKGjewXNpjD74sJTjJ1iNgTH, DXCzguRGhTGvFm9hdVsDkFi4S3n6W2yrNeUjrFN8tkvL, D23NCAVxinE53BTemguZCheAqCdMGfNTUzWdoWvq4Xj, 8dHEsm9aLBt7q6zu3ESfRXkS2eCwkbbzzynfd2QxDzms, 5fnyGEnVu3nyMrUysGQLXz38QH51VNtmYGSA99197xCX, E2cy4hqcUpdyMpx3TuHKpdW2cJZ3cTSthk4jfqJryt6B, G7gAgJpRHnRvFhrUMA5khWMqHJ3tpWVWdpsBvCq6w6MY, A1voPbfnmCq8UBNQTBKnZ3Xbhs2x4cS2Gx2b2wJtqCh1, HsWUiXARLPhYitGMapLYyMdV7k27kW2xzy9Z6L77jKBC, AcjhWohnu7vYMdu4Yha63XZupqMKVVnrWmt1F57ScXhG, BLZtwHMTMgnZJdhJQxQaksgJgteXFFDBrA13ywWagji4, C4eTa4tqvvzpTsp9pa5NKAbeDXJs6sHWS5BfxGB44Xex, 7K32uTNK2zJwp5WTt4t57qJMf1JnHBq2HcSkc4oV5sQb, DVnKs7XAL9au7cWrTT335gZ3agJVwrqeSVnSWANo1SJG, HgmPwzNcY85HfrN3bYiqaypb6Nmf7ayaZEaivGY37913, HELPwwfg5W9LmXv3axe43EY1YGJjfVf3CcVjA8BZM82P, 8FRFYPcwBan1KBKR6HuPy152L7pr3ePVYVxXXnWzPjEd, 7scarR3Z5obfefZr8bPKYoMNipua43K35AJAc1YchQBK, GRJohz4qkCV6YNjcSb3mRBYbtcTcN7GLG2cm6ffZqmW3, E9qZxXtwWT5FuwsXHLjA4cjJyyeYb3ixHxBSrJJDzPwx, 8aZtHhTNFhVWp4fV3dUfBwsKKBjqzHDwpTZRbpeqo7vo, 37Gr1zVPr79E3AdPFj8EMyKZYt7Bnz3VWKjdFctQC8fL, CKYDvsLjwp6tPXfYxLmBv9LtX3FPpaAPj8kw1Gqy7s8Y, 4JZsGW4WUSjAjH4joCaAAVnNi5ERfHr93YUDxmHZpDM7, 5KFXF9DS2ETQVfUTAfMygw6LNbqiXWXYr4y2k1kYr9bA, 9U4fqWRd3kcUHEX2jt1kFwF2dSXLnz9RA6B9W656Skbv, CPPVEbGFbX3XAThetvfveCE1vYLWUwwJGT7DxkPAWb8D, FNH1XmR5WgK7CH7W7YdcfxtdgaKFueKtqaggVr3CnY7M, 9cZua5prTSEfednQQc9RkEPpbKDCh1AwnTzv3hE1eq3i, HWvSRgESdWKDccWN91iRVQLN4rRyuCbuAHVWtPR1cJ1C, GXfJaLrWgQbiuutiLyN7ijRgBvAvunJy7bYzaV562VWP, HZCUCLqV3P7QqG1oskLLMJW28zuckhxmRzEQ7UWaH2U4, Bdd4XhquueXBB7aZXVYUn1XBdJ18G7Wx3LUe6aKkmXEV, 3Wyqj2cgKYK2sSSb3wVv3wJ5yD3yigV8iLLttkZfKn8d, DFpi7mgmChYV9whs4uEtioFG1R2WF4TpGd1zcXMjGwF3, 2GAdxV8QafdRnkTwy9AuX8HvVcNME6JqK2yANaDunhXp, 4vXPjSaZfydRqhnM85uFqDWqYcFyA744R2tjZQN8Nff4, 3xUTkgPKNJZ3dkpDMV8zWV34BkmvKanguKipv6M9x2Mt, 55nmQ8gdWpNW5tLPoBPsqDkLm1W24cmY5DbMMXZKSP8U, FdC2FZ4geXeEh4pTPXWrNH4nqsriB3xD3GME9HPD8CVR, FQFrdHAhKFP9R5R6JkJPtVJhLDivDia4cNNUcL6Eja6j, EngVeJ8w7soeVvKwypuSutnXFPSWDLMq3Vw5wuAdSGjf, 2CGskjnksG9YwAFMJkPDwsKx1iRAXJSzfpAxyoWzGj6M, 3kWT2K2HfxrspLFoJhKUAio3QF85EuTemJKTUcPEjm7m, B6ZramCQhQWcq4Vxo3H5y1MYvo37wRZiAwKk88qaYNiF, 4ZtE2XX6oQThPpdjwKXVMphTTZctbWwYxmcCV6xR11RT, CM1c6z3pRNgHFcfZG4z3wE31jaR8c4gCYQBJVEoCUyq8, FEq9FL3hzRDMtL8DinPAaeJpb28GBZYTpTeRcRyHSrGA, 5dLMRyPWx6rdPGZpZ7uuZiqry96dUT5yz48u62Gzugi6, A8Lv2ZPKKSBFiAiepFsmCBvWEBSVGzuKxSLVt9z62Bqt, 3mx22d1aJLazEutJyHVszdwyLJcrRo26EKB4AWDbRxRc, HVK74e6bD6cwDrK28hijNxiHZkvyXCXS8bd41tH6QwTk, 8ZgmpBG5ixt4LVRQEK538hsKTsJBgmFFH5L6X5e9iPTd, HzPFqFKsGRT3Yvd5Wgfng16c8q6e1bDe3W48fZbuuS9Q, 7arfejY2YxX9QrmzHrhu3rG3HofjMqKtfBzQLf8s3Wop, 7T5ZekSsBSgLNKVzQmCRQ5iqL5ycprREa1tz3GYmb4eT, A5TXyVrR7WwfNf2RjoN1W4Dw5CuuMDiLV9e77pWhmwAP, 7F2vcJca5ewzdJUNcVMKCLVYneq6CX9JFMH1U7JeVG5, HfxzFiP19ymtxHagP4Zpga2zVo6ZgivpK6VkBKDowHRr, DhMuXF3UqZvi3GhdrAMVyEQ7pW4prM8DkW54scYXo9Ke, CbhvvtosVdVwZ8GVrBqgYT3JrXLh8JRqgKpimhnZw31h, 8zH1mRkic3WDpUkSgtq1geCXXh4CLVfLrEi2TEqdTgFS, xfCpo4ouRs5BP3WY5BdWhbr41pQxYGcXxz1sFyzPsZr, Gb9j79QFprtbY4sieaLZGjQr4a5ifFxGdnU2qkjPbBJQ, 5vfvM4qv8UERxSU4qjKhcyJYgfvBwxM3zotkbyXg5z4z, FmVd4YWnsiau4JLHYxpRW63uutA2sPKpKYGiwB3hBkwt, EG8D25QxDJ6nbD3oBpu4tPDvihriy9mFiPq3CxCGFiPF, 9B4oF52Web2deG8jdbNbBuM8DjiFyLYM6CTid3dmQkQ6, ACv5dTk7THbmUpHYGhgPzMhWr7oqHSkuPJpa5RfvmG5H, 9oKrJ9iiEnCC7bewcRFbcdo4LKL2PhUEqcu8gH2eDbVM, BBvV71ncxGrMDjmKTkcvcjipzu3bv6ExyVPPuRxAPtrC, 5KZRD6hDCZd1dgM9rJLweFFRkHu3qAgBhHgQ96Wz1VSn, 518q2YT5TjpwZM3sLSTk58VVmdYkF86abh7GGyoUaHZ, HDitfpmCcy8WyJgNCQTrnZ8r71Nn7t7SjmVHnRwumGZi, DEZAHY54DgLq9Md8CyxBgNCe5hxDQi7fJaSE8jymtazr, markiLNTC3FuWYGXKz8h9XpbwJbVQhzuV5U3bfpPc64, H4Dgb3KyCuYWKT8yKtp8qbY49cvaqZcisa2GDnroFsv6, FkYhpz7HSGQJvA6apj1BKoUfytQvWseLfSUrE3zjvkQb, 7EucomZSKvQdiZLvra8hLszL1kRYiGewxyMJnyyzdbH7, 4XWxphAh1Ji9p3dYMNRNtW3sbmr5Z1cvsGyJXJx5Jvfy, HKu753Hd2F1nWLPvcNZHX6RAGSXkg6AtywiVvRqDXxcP, 9QxCLckBiJc783jnMvXZubK4wH86Eqqvashtrwvcsgkv, 7JFfCpPEodnt6SWY41ePBRXR6LUGiKhLSKJNw9ZYjdah, Dg5E8ktH4GWfKL1vuVTdqZJEkAEgtV8LqmSXyLJuZ3q1, BYxEvmSwA1o3vJGhooJoReSjrzou8A9h1TZX9S4xomHP, AgFQkQe2Em2GUkDD85qPmHrvybnaXKMa7anSNdCunnM4, AK7ZZx2sdo39coZN5FsPdae2xNGqVKHX2TWJixmY4ecX},
slot: 52742164,
epoch: Some(134)
[2020-12-15T01:14:54.710187608Z INFO solana_metrics::metrics] datapoint: replay-loop-timing-stats total_elapsed_us=1024000i compute_bank_stats_elapsed=12095i select_vote_and_reset_forks_elapsed=17i start_leader_elapsed=746i reset_bank_elapsed=120i voting_elapsed=13676i select_forks_elapsed=33i compute_slot_stats_elapsed=6i generate_new_bank_forks_elapsed=9975i replay_active_banks_elapsed=50783i reset_duplicate_slots_elapsed=8588i
[2020-12-15T01:14:54.718497608Z INFO solana_metrics::metrics] datapoint: poh-service ticks=128i hashes=1599872i elapsed_ms=633i
[2020-12-15T01:14:54.823113866Z INFO solana_metrics::metrics] datapoint: replay_stage-new_leader slot=52742167i leader="71bhKKL89U3dNHzuZVZ7KarqV6XtHEgjXjvJTsguD11B"
[2020-12-15T01:14:54.927552103Z INFO solana_metrics::counter] COUNTER:{"name": "bank-process_transactions", "counts": 239986046, "samples": 226288000, "now": 1607994894927, "events": 1}
[2020-12-15T01:14:54.931453775Z INFO solana_core::replay_stage] new fork:52742167 parent:52742164 (leader) root:52742125
[2020-12-15T01:14:54.931463582Z INFO solana_metrics::metrics] datapoint: replay_stage-new_leader slot=52742167i leader="71bhKKL89U3dNHzuZVZ7KarqV6XtHEgjXjvJTsguD11B"
[2020-12-15T01:14:54.931491365Z INFO solana_metrics::metrics] datapoint: replay_stage-my_leader_slot slot=52742167i
[2020-12-15T01:14:54.932834962Z INFO solana_metrics::metrics] datapoint: bank-new_from_parent-heights slot_height=52742167i block_height=42225191i
[2020-12-15T01:14:54.937800131Z INFO solana_metrics::metrics] datapoint: bank-timestamp get_timestamp_estimate_us=455i
[2020-12-15T01:14:54.937817167Z INFO solana_metrics::metrics] datapoint: bank-timestamp-correction slot=52742167i from_genesis=1602191934i corrected=1602191934i ancestor_timestamp=1602191932i
[2020-12-15T01:14:54.939971799Z INFO solana_metrics::metrics] datapoint: cluster_info-num_nodes live_count=463i broadcast_count=480i
[2020-12-15T01:14:54.964777003Z INFO solana_metrics::metrics] datapoint: optimistic_slot_elapsed average_elapsed_ms=1038i
[2020-12-15T01:14:54.964800683Z INFO solana_metrics::metrics] datapoint: optimistic_slot slot=52742164i
[2020-12-15T01:14:54.977207902Z INFO solana_metrics::counter] COUNTER:{"name": "banking_stage-dropped_tx_before_forwarding", "counts": 1417, "samples": 2913000, "now": 1607994894977, "events": 0}
[2020-12-15T01:14:54.987642722Z INFO solana_metrics::counter] COUNTER:{"name": "bank-process_transactions-txs", "counts": 200671142, "samples": 224408000, "now": 1607994894987, "events": 1}
[2020-12-15T01:14:54.987676664Z INFO solana_metrics::counter] COUNTER:{"name": "bank-process_transactions-sigs", "counts": 238101774, "samples": 224408000, "now": 1607994894987, "events": 1}
[2020-12-15T01:14:54.994043185Z ERROR solana_metrics::counter] COUNTER:{"name": "bank-process_transactions-error_count", "counts": 39315254, "samples": 39254000, "now": 1607994894994, "events": 1}
[2020-12-15T01:14:55.002405913Z ERROR solana_metrics::counter] COUNTER:{"name": "bank-process_transactions-error-instruction_error", "counts": 37429322, "samples": 37373000, "now": 1607994895002, "events": 1}
[2020-12-15T01:14:55.007536572Z INFO solana_metrics::counter] COUNTER:{"name": "bank-process_transactions", "counts": 239987063, "samples": 226289000, "now": 1607994895007, "events": 1}
[2020-12-15T01:14:55.007803758Z WARN solana_metrics::counter] COUNTER:{"name": "banking_stage-record_transactions", "counts": 1077762, "samples": 1033000, "now": 1607994895007, "events": 1}
[2020-12-15T01:14:55.020509250Z INFO solana_core::poh_recorder] poh_record: max_tick_height 3375498752 reached, clearing working_bank 52742167
[2020-12-15T01:14:55.037094621Z INFO solana_metrics::counter] COUNTER:{"name": "vote-native", "counts": 238079001, "samples": 238079000, "now": 1607994895037, "events": 1}
[2020-12-15T01:14:55.046714469Z INFO solana_metrics::metrics] datapoint: broadcast-process-shreds-stats slot=52742167i shredding_time=30384i receive_time=146i num_data_shreds=242i slot_broadcast_time=104271i
[2020-12-15T01:14:55.046906024Z INFO solana_metrics::metrics] datapoint: shred_insert_is_full total_time_ms=108i slot=52742167i last_index=241i
[2020-12-15T01:14:55.047274659Z INFO solana_metrics::metrics] datapoint: broadcast-insert-shreds-stats slot=52742167i end_to_end_elapsed=109406i insert_shreds_elapsed=23048i num_shreds=484i
[2020-12-15T01:14:55.047738669Z INFO solana_core::replay_stage] bank frozen: 52742167
[2020-12-15T01:14:55.047749606Z INFO solana_metrics::metrics] datapoint: replay-slot-stats slot=52742167i fetch_entries_time=0i fetch_entries_fail_time=0i entry_poh_verification_time=0i entry_transaction_verification_time=0i replay_time=0i replay_total_elapsed=40i total_entries=0i total_shreds=0i
[2020-12-15T01:14:55.047877850Z INFO solana_metrics::metrics] datapoint: collect_rent_eagerly accounts=7i
[2020-12-15T01:14:55.060217201Z INFO solana_runtime::bank] bank frozen: 52742167 hash: BVo19SyzSHWkkcch6kgcZAts2m6NHvEtHdNKPsmn8Ra3 accounts_delta: 4gqk9gvntYHvhEnHi3b1QXBCK71PPEcuwhAqaexePRxH signature_count: 558 last_blockhash: 6mJGdEka4xo6WnuR3BTqdHQeaAqSzDUnjEzGZDusgfdc capitalization: 1293979802487593051
[2020-12-15T01:14:55.060253632Z INFO solana_runtime::bank] accounts hash slot: 52742167 stats: BankHashStats { num_updated_accounts: 1461, num_removed_accounts: 0, num_lamports_stored: 911385404894551801, total_data_len: 1871679, num_executable_accounts: 0 }
[2020-12-15T01:14:55.071155392Z INFO solana_metrics::metrics] datapoint: tower-observed slot=52742164i root=52742124i
[2020-12-15T01:14:55.074598173Z INFO solana_core::replay_stage] 71bhKKL89U3dNHzuZVZ7KarqV6XtHEgjXjvJTsguD11B slot_weight: 52742167 0 0 52742164
[2020-12-15T01:14:55.074616486Z INFO solana_metrics::metrics] datapoint: best_slot slot=52742167i best_slot=52742167i
[2020-12-15T01:14:55.074635036Z INFO solana_core::replay_stage] validator fork confirmed 52742163 2224ms
[2020-12-15T01:14:55.074637181Z INFO solana_metrics::metrics] datapoint: bank_weight slot=52742167i weight="0"
[2020-12-15T01:14:55.074643822Z INFO solana_metrics::metrics] datapoint: validator-confirmation duration_ms=2224i
[2020-12-15T01:14:55.074646073Z INFO solana_core::replay_stage] voting: 52742167 0
[2020-12-15T01:14:55.074675358Z INFO solana_metrics::metrics] datapoint: tower-vote latest=52742167i root=52742125i
[2020-12-15T01:14:55.079207489Z INFO solana_core::replay_stage] vote bank: Some((52742167, SameFork)) reset bank: 52742167
[2020-12-15T01:14:55.079271840Z INFO solana_metrics::metrics] datapoint: blocks_produced num_blocks_on_fork=534091i num_dropped_blocks_on_fork=110207i
[2020-12-15T01:14:55.079295230Z INFO solana_core::poh_recorder] reset poh from: 9pjyq9cEtnyb7TbWd1XFymJQv945x3BS3x76hTWxTXyp,3375498763,52742167 to: 6mJGdEka4xo6WnuR3BTqdHQeaAqSzDUnjEzGZDusgfdc,52742167
[2020-12-15T01:14:55.079310504Z INFO solana_core::replay_stage] 71bhKKL89U3dNHzuZVZ7KarqV6XtHEgjXjvJTsguD11B reset PoH to tick 3375498752 (within slot 52742167). My next leader slot is 52743164
[2020-12-15T01:14:55.082722306Z INFO solana_metrics::metrics] datapoint: block-commitment-cache aggregate-commitment-ms=3i highest-confirmed-root=52742124i highest-confirmed-slot=52742164i
[2020-12-15T01:14:55.085081345Z INFO solana_metrics::metrics] datapoint: broadcast-transmit-shreds-stats slot=52742167i end_to_end_elapsed=147204i transmit_elapsed=20090i send_mmsg_elapsed=6013i get_peers_elapsed=557004i num_shreds=484i shred_select=8343i
[2020-12-15T01:14:55.252315203Z INFO solana_metrics::metrics] datapoint: do_shrink_slot_time storage_read_elapsed=1i index_read_elapsed=0i find_alive_elapsed=0i create_and_insert_store_elapsed=0i store_accounts_elapsed=0i update_index_elapsed=0i handle_reclaims_elapsed=0i write_storage_elapsed=0i rewrite_elapsed=0i drop_storage_entries_elapsed=0i
[2020-12-15T01:14:55.275733784Z INFO solana_metrics::metrics] datapoint: shred_insert_is_full total_time_ms=969i slot=52742168i last_index=375i
[2020-12-15T01:14:55.285482950Z INFO solana_core::replay_stage] new fork:52742169 parent:52742164 root:52742125
[2020-12-15T01:14:55.287015224Z INFO solana_metrics::metrics] datapoint: bank-new_from_parent-heights slot_height=52742169i block_height=42225191i
[2020-12-15T01:14:55.293811856Z INFO solana_metrics::metrics] datapoint: bank-timestamp get_timestamp_estimate_us=659i
[2020-12-15T01:14:55.293832457Z INFO solana_metrics::metrics] datapoint: bank-timestamp-correction slot=52742169i from_genesis=1602191935i corrected=1602191935i ancestor_timestamp=1602191932i
[2020-12-15T01:14:55.327720896Z WARN solana_ledger::blockstore_processor] Too few entry ticks found in slot: 52742168
[2020-12-15T01:14:55.327747345Z WARN solana_ledger::blockstore_processor] InvalidTickCount, slot: 52742168, entry len: 0, tick_height: 3375498815, last entry: GEW2VETiZqMnU57kUcLY4aCwNiF8AgNPouYuez2FiiF1, last_blockhash: 6KGZXsbmzjPNyuDdNjVi8NmmbvwmhxstQjGhT58XsDPa, shred_index: 1, slot_full: true
[2020-12-15T01:14:55.327766836Z WARN solana_core::replay_stage] Fatal replay error in slot: 52742168, err: InvalidBlock(InvalidTickCount)
[2020-12-15T01:14:55.327802269Z ERROR solana_metrics::metrics] datapoint: replay-stage-mark_dead_slot error="error: InvalidBlock(InvalidTickCount)" slot=52742168i
[2020-12-15T01:14:55.328189437Z INFO solana_metrics::metrics] datapoint: cluster_info_stats entrypoint=0i entrypoint2=0i push_vote_read=77i vote_process_push=0i get_votes=17078i get_accounts_hash=0i all_tvu_peers=0i tvu_peers=363255i new_push_requests_num=5098i table_size=64594i purged_values_size=103882i failed_inserts_size=4234i
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment