Skip to content

Instantly share code, notes, and snippets.

@yorickdowne
Last active February 14, 2023 10:52
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save yorickdowne/27d4a96174562377d86e4df5d8de654f to your computer and use it in GitHub Desktop.
Save yorickdowne/27d4a96174562377d86e4df5d8de654f to your computer and use it in GitHub Desktop.
A systemd unit file for clean Geth restart

Overview

Geth, and other execution clients such as Erigon, Nethermind, and Besu, take some time on shutdown to cleanly close their database. When not given that time, they will need to resync upon restart, often for hours. This gist describes the systemd service file parameter to give them up to 5 minutes of time for a clean shutdown.

Steps

  • Find your service file. It may be called /etc/systemd/system/geth.service or /etc/systemd/system/eth1.service.
  • Edit it using sudo nano FILENAME, and add TimeoutStopSec=300 to the [Service] section. Here is an example of a service file after the change:
[Unit]
Description=Go Ethereum Client
After=network.target
Wants=network.target

[Service]
User=goeth
Group=goeth
Type=simple
Restart=always
RestartSec=5
TimeoutStopSec=300
ExecStart=geth --http --datadir /var/lib/goethereum

[Install]
WantedBy=default.target
  • Tell systemd that the file has changed: sudo systemctl daemon-reload

Done! Next time you stop Geth, or any other client you made this change for, it should cleany persist the database to disk and start up without a lengthy resync.

@PanosChtz
Copy link

PanosChtz commented Aug 22, 2022

I put the 3 minutes timeout but geth still seems to shutdown in a non-proper way. Any suggestions? Is it maybe because I am running it on a RPi4 which needs more time? See log below..

Aug 22 15:12:42 validator geth[929]: INFO [08-22|15:12:42.058] Imported new chain segment               blocks=1   txs=313   mgas=29.993  elapsed=18.829s     mgasps=1.593   number=15,390,966 hash=59d033..d3052f age=3m49s    dirty=255.12MiB
Aug 22 15:12:51 validator geth[929]: INFO [08-22|15:12:51.710] Deep froze chain segment                 blocks=7   elapsed=176.215ms   number=15,300,966 hash=362838..f1bf8d
Aug 22 15:12:54 validator geth[929]: INFO [08-22|15:12:54.715] Imported new chain segment               blocks=1   txs=390   mgas=29.988  elapsed=12.656s     mgasps=2.369   number=15,390,967 hash=9a73e6..4e603f age=3m35s    dirty=255.37MiB
Aug 22 15:13:34 validator geth[929]: INFO [08-22|15:13:34.167] Imported new chain segment               blocks=1   txs=311   mgas=29.984  elapsed=39.452s     mgasps=0.760   number=15,390,968 hash=7ae270..0ea4db age=3m28s    dirty=255.18MiB
Aug 22 15:13:34 validator geth[929]: INFO [08-22|15:13:34.174] Downloader queue stats                   receiptTasks=0 blockTasks=0  itemSize=92.37KiB  throttle=2839
Aug 22 15:13:34 validator geth[929]: INFO [08-22|15:13:34.483] Unindexed transactions                   blocks=12  txs=2599  tail=13,040,969 elapsed=310.257ms
Aug 22 15:13:51 validator geth[929]: INFO [08-22|15:13:51.985] Deep froze chain segment                 blocks=2   elapsed=268.559ms   number=15,300,968 hash=c2dc09..7a7674
Aug 22 15:13:53 validator geth[929]: INFO [08-22|15:13:53.971] Imported new chain segment               blocks=1   txs=496   mgas=29.981  elapsed=18.289s     mgasps=1.639   number=15,390,969 hash=5bf2a7..956190 age=3m29s    dirty=254.42MiB
Aug 22 15:13:53 validator geth[929]: INFO [08-22|15:13:53.981] Unindexed transactions                   blocks=1   txs=17    tail=13,040,970 elapsed=7.850ms
Aug 22 15:13:55 validator systemd[1]: Stopping Go Ethereum Client...
Aug 22 15:13:55 validator geth[929]: INFO [08-22|15:13:55.062] Got interrupt, shutting down...
Aug 22 15:13:55 validator geth[929]: INFO [08-22|15:13:55.104] HTTP server stopped                      endpoint=127.0.0.1:8545
Aug 22 15:13:55 validator geth[929]: INFO [08-22|15:13:55.111] IPC endpoint closed                      url=/var/lib/goethereum/geth.ipc
Aug 22 15:14:08 validator geth[929]: INFO [08-22|15:14:08.898] Imported new chain segment               blocks=1   txs=1130  mgas=29.993  elapsed=14.909s     mgasps=2.012   number=15,390,970 hash=aac1cd..1d3872 age=2m28s    dirty=255.41MiB ignored=2
Aug 22 15:14:09 validator geth[929]: INFO [08-22|15:14:09.321] Unindexed transactions                   blocks=1   txs=41    tail=13,040,971 elapsed=79.423ms
Aug 22 15:14:19 validator geth[929]: INFO [08-22|15:14:19.333] Looking for peers                        peercount=2  tried=31 static=0
Aug 22 15:14:19 validator geth[929]: WARN [08-22|15:14:19.425] Snapshot extension registration failed   peer=cc4b99d8 err="peer connected on snap without compatible eth support"
Aug 22 15:14:21 validator geth[929]: WARN [08-22|15:14:21.473] Snapshot extension registration failed   peer=7a77d37c err="peer connected on snap without compatible eth support"
Aug 22 15:14:24 validator geth[929]: WARN [08-22|15:14:24.356] Snapshot extension registration failed   peer=5450e190 err="peer connected on snap without compatible eth support"
Aug 22 15:14:24 validator geth[929]: WARN [08-22|15:14:24.904] Snapshot extension registration failed   peer=569f3be3 err="peer connected on snap without compatible eth support"
Aug 22 15:14:30 validator geth[929]: INFO [08-22|15:14:30.956] Looking for peers                        peercount=2  tried=194 static=0
Aug 22 15:14:41 validator geth[929]: INFO [08-22|15:14:41.354] Looking for peers                        peercount=2  tried=92  static=0
Aug 22 15:14:44 validator geth[929]: WARN [08-22|15:14:44.124] Snapshot extension registration failed   peer=7501eaab err="peer connected on snap without compatible eth support"
Aug 22 15:14:50 validator geth[929]: WARN [08-22|15:14:50.904] Snapshot extension registration failed   peer=cbd3f861 err="peer connected on snap without compatible eth support"
Aug 22 15:14:51 validator geth[929]: INFO [08-22|15:14:51.432] Looking for peers                        peercount=2  tried=159 static=0
Aug 22 15:14:52 validator geth[929]: INFO [08-22|15:14:52.607] Deep froze chain segment                 blocks=2   elapsed=599.880ms   number=15,300,970 hash=7429b4..74b931
Aug 22 15:15:01 validator geth[929]: INFO [08-22|15:15:01.470] Looking for peers                        peercount=1  tried=102 static=0
Aug 22 15:15:11 validator geth[929]: INFO [08-22|15:15:11.485] Looking for peers                        peercount=1  tried=95  static=0
Aug 22 15:15:21 validator geth[929]: WARN [08-22|15:15:21.099] Snapshot extension registration failed   peer=98998f1a err="peer connected on snap without compatible eth support"
Aug 22 15:15:21 validator geth[929]: INFO [08-22|15:15:21.715] Looking for peers                        peercount=1  tried=17  static=0
Aug 22 15:15:23 validator geth[929]: WARN [08-22|15:15:23.414] Snapshot extension registration failed   peer=c85be8b4 err="peer connected on snap without compatible eth support"
Aug 22 15:15:28 validator geth[929]: WARN [08-22|15:15:28.813] Snapshot extension registration failed   peer=cbd3f861 err="peer connected on snap without compatible eth support"
Aug 22 15:15:31 validator geth[929]: INFO [08-22|15:15:31.889] Looking for peers                        peercount=1  tried=221 static=0
Aug 22 15:15:35 validator geth[929]: WARN [08-22|15:15:35.320] Snapshot extension registration failed   peer=5450e190 err="peer connected on snap without compatible eth support"
Aug 22 15:15:42 validator geth[929]: INFO [08-22|15:15:42.038] Looking for peers                        peercount=2  tried=116 static=0
Aug 22 15:15:42 validator geth[929]: WARN [08-22|15:15:42.722] Snapshot extension registration failed   peer=5271cee3 err="peer connected on snap without compatible eth support"
Aug 22 15:15:52 validator geth[929]: INFO [08-22|15:15:52.181] Looking for peers                        peercount=2  tried=73  static=0
Aug 22 15:15:54 validator geth[929]: WARN [08-22|15:15:54.439] Snapshot extension registration failed   peer=4dfa8e8a err="peer connected on snap without compatible eth support"
Aug 22 15:16:02 validator geth[929]: INFO [08-22|15:16:02.192] Looking for peers                        peercount=1  tried=86  static=0
Aug 22 15:16:12 validator geth[929]: INFO [08-22|15:16:12.237] Looking for peers                        peercount=1  tried=110 static=0
Aug 22 15:16:13 validator geth[929]: WARN [08-22|15:16:13.712] Snapshot extension registration failed   peer=b504b033 err="peer connected on snap without compatible eth support"
Aug 22 15:16:14 validator geth[929]: WARN [08-22|15:16:14.028] Snapshot extension registration failed   peer=a90b10c7 err="peer connected on snap without compatible eth support"
Aug 22 15:16:22 validator geth[929]: INFO [08-22|15:16:22.429] Looking for peers                        peercount=1  tried=93  static=0
Aug 22 15:16:25 validator geth[929]: WARN [08-22|15:16:25.190] Snapshot extension registration failed   peer=8ffb8a85 err="peer connected on snap without compatible eth support"
Aug 22 15:16:32 validator geth[929]: INFO [08-22|15:16:32.447] Looking for peers                        peercount=1  tried=140 static=0
Aug 22 15:16:43 validator geth[929]: INFO [08-22|15:16:43.264] Looking for peers                        peercount=2  tried=10  static=0
Aug 22 15:16:46 validator geth[929]: WARN [08-22|15:16:46.765] Snapshot extension registration failed   peer=7501eaab err="peer connected on snap without compatible eth support"
Aug 22 15:16:47 validator geth[929]: WARN [08-22|15:16:47.059] Snapshot extension registration failed   peer=e87896a1 err="peer connected on snap without compatible eth support"
Aug 22 15:16:53 validator geth[929]: INFO [08-22|15:16:53.380] Looking for peers                        peercount=1  tried=94  static=0
Aug 22 15:16:55 validator systemd[1]: geth.service: State 'stop-sigterm' timed out. Killing.
Aug 22 15:16:55 validator systemd[1]: geth.service: Killing process 929 (geth) with signal SIGKILL.
Aug 22 15:16:55 validator systemd[1]: geth.service: Killing process 968 (geth) with signal SIGKILL.
Aug 22 15:16:55 validator systemd[1]: geth.service: Killing process 969 (geth) with signal SIGKILL.
Aug 22 15:16:55 validator systemd[1]: geth.service: Killing process 1008 (n/a) with signal SIGKILL.
Aug 22 15:16:55 validator systemd[1]: geth.service: Killing process 1071 (geth) with signal SIGKILL.
Aug 22 15:16:55 validator systemd[1]: geth.service: Killing process 1072 (n/a) with signal SIGKILL.
Aug 22 15:16:55 validator systemd[1]: geth.service: Killing process 1183 (n/a) with signal SIGKILL.
Aug 22 15:16:55 validator systemd[1]: geth.service: Killing process 1185 (n/a) with signal SIGKILL.
Aug 22 15:16:55 validator systemd[1]: geth.service: Killing process 1186 (n/a) with signal SIGKILL.
Aug 22 15:16:55 validator systemd[1]: geth.service: Killing process 1187 (geth) with signal SIGKILL.
Aug 22 15:16:55 validator systemd[1]: geth.service: Killing process 1188 (n/a) with signal SIGKILL.
Aug 22 15:16:55 validator systemd[1]: geth.service: Killing process 1189 (geth) with signal SIGKILL.
Aug 22 15:16:55 validator systemd[1]: geth.service: Killing process 1547 (n/a) with signal SIGKILL.
Aug 22 15:16:55 validator systemd[1]: geth.service: Killing process 6274 (n/a) with signal SIGKILL.
Aug 22 15:16:55 validator systemd[1]: geth.service: Killing process 65752 (n/a) with signal SIGKILL.
Aug 22 15:16:55 validator systemd[1]: geth.service: Killing process 143508 (n/a) with signal SIGKILL.
Aug 22 15:16:55 validator systemd[1]: geth.service: Killing process 196838 (geth) with signal SIGKILL.
Aug 22 15:16:59 validator systemd[1]: geth.service: Main process exited, code=killed, status=9/KILL
Aug 22 15:16:59 validator systemd[1]: geth.service: Failed with result 'timeout'.
Aug 22 15:16:59 validator systemd[1]: Stopped Go Ethereum Client.
Aug 22 15:17:22 validator systemd[1]: Started Go Ethereum Client.
Aug 22 15:17:23 validator geth[316890]: INFO [08-22|15:17:23.296] Starting Geth on Ethereum mainnet...
Aug 22 15:17:23 validator geth[316890]: INFO [08-22|15:17:23.306] Maximum peer count                       ETH=30 LES=0 total=30
Aug 22 15:17:23 validator geth[316890]: INFO [08-22|15:17:23.312] Smartcard socket not found, disabling    err="stat /run/pcscd/pcscd.comm: no such file or directory"
Aug 22 15:17:23 validator geth[316890]: INFO [08-22|15:17:23.333] Set global gas cap                       cap=50,000,000
Aug 22 15:17:23 validator geth[316890]: INFO [08-22|15:17:23.338] Allocated trie memory caches             clean=153.00MiB dirty=256.00MiB
Aug 22 15:17:23 validator geth[316890]: INFO [08-22|15:17:23.340] Allocated cache and file handles         database=/var/lib/goethereum/geth/chaindata cache=512.00MiB handles=262,144
Aug 22 15:18:03 validator geth[316890]: INFO [08-22|15:18:03.176] Found legacy ancient chain path          location=/var/lib/goethereum/geth/chaindata/ancient
Aug 22 15:18:03 validator geth[316890]: INFO [08-22|15:18:03.247] Opened ancient database                  database=/var/lib/goethereum/geth/chaindata/ancient readonly=false
Aug 22 15:18:03 validator geth[316890]: INFO [08-22|15:18:03.272]
Aug 22 15:18:03 validator geth[316890]: INFO [08-22|15:18:03.272] ---------------------------------------------------------------------------------------------------------------------------------------------------------
Aug 22 15:18:03 validator geth[316890]: INFO [08-22|15:18:03.272] Chain ID:  1 (mainnet)
Aug 22 15:18:03 validator geth[316890]: INFO [08-22|15:18:03.272] Consensus: Beacon (proof-of-stake), merging from Ethash (proof-of-work)
Aug 22 15:18:03 validator geth[316890]: INFO [08-22|15:18:03.273]
Aug 22 15:18:03 validator geth[316890]: INFO [08-22|15:18:03.273] Pre-Merge hard forks:
Aug 22 15:18:03 validator geth[316890]: INFO [08-22|15:18:03.273]  - Homestead:                   1150000  (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/homestead.md)
Aug 22 15:18:03 validator geth[316890]: INFO [08-22|15:18:03.273]  - DAO Fork:                    1920000  (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/dao-fork.md)
Aug 22 15:18:03 validator geth[316890]: INFO [08-22|15:18:03.273]  - Tangerine Whistle (EIP 150): 2463000  (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/tangerine-whistle.md)
Aug 22 15:18:03 validator geth[316890]: INFO [08-22|15:18:03.273]  - Spurious Dragon/1 (EIP 155): 2675000  (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/spurious-dragon.md)
Aug 22 15:18:03 validator geth[316890]: INFO [08-22|15:18:03.273]  - Spurious Dragon/2 (EIP 158): 2675000  (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/spurious-dragon.md)
Aug 22 15:18:03 validator geth[316890]: INFO [08-22|15:18:03.273]  - Byzantium:                   4370000  (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/byzantium.md)
Aug 22 15:18:03 validator geth[316890]: INFO [08-22|15:18:03.273]  - Constantinople:              7280000  (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/constantinople.md)
Aug 22 15:18:03 validator geth[316890]: INFO [08-22|15:18:03.273]  - Petersburg:                  7280000  (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/petersburg.md)
Aug 22 15:18:03 validator geth[316890]: INFO [08-22|15:18:03.273]  - Istanbul:                    9069000  (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/istanbul.md)
Aug 22 15:18:03 validator geth[316890]: INFO [08-22|15:18:03.273]  - Muir Glacier:                9200000  (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/muir-glacier.md)
Aug 22 15:18:03 validator geth[316890]: INFO [08-22|15:18:03.273]  - Berlin:                      12244000 (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/berlin.md)
Aug 22 15:18:03 validator geth[316890]: INFO [08-22|15:18:03.273]  - London:                      12965000 (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/london.md)
Aug 22 15:18:03 validator geth[316890]: INFO [08-22|15:18:03.273]  - Arrow Glacier:               13773000 (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/arrow-glacier.md)
Aug 22 15:18:03 validator geth[316890]: INFO [08-22|15:18:03.273]  - Gray Glacier:                15050000 (https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/gray-glacier.md)
Aug 22 15:18:03 validator geth[316890]: INFO [08-22|15:18:03.273]
Aug 22 15:18:03 validator geth[316890]: INFO [08-22|15:18:03.273] Merge configured:
Aug 22 15:18:03 validator geth[316890]: INFO [08-22|15:18:03.273]  - Hard-fork specification:    https://github.com/ethereum/execution-specs/blob/master/network-upgrades/mainnet-upgrades/paris.md
Aug 22 15:18:03 validator geth[316890]: INFO [08-22|15:18:03.273]  - Network known to be merged: false
Aug 22 15:18:03 validator geth[316890]: INFO [08-22|15:18:03.273]  - Total terminal difficulty:  58750000000000000000000
Aug 22 15:18:03 validator geth[316890]: INFO [08-22|15:18:03.273]  - Merge netsplit block:       <nil>
Aug 22 15:18:03 validator geth[316890]: INFO [08-22|15:18:03.274] ---------------------------------------------------------------------------------------------------------------------------------------------------------
Aug 22 15:18:03 validator geth[316890]: INFO [08-22|15:18:03.274]
Aug 22 15:18:15 validator geth[316890]: INFO [08-22|15:18:15.394] Disk storage enabled for ethash caches   dir=/var/lib/goethereum/geth/ethash count=3
Aug 22 15:18:15 validator geth[316890]: INFO [08-22|15:18:15.394] Disk storage enabled for ethash DAGs     dir=/home/goeth/.ethash             count=2
Aug 22 15:18:15 validator geth[316890]: INFO [08-22|15:18:15.404] Initialising Ethereum protocol           network=1 dbversion=8
Aug 22 15:18:16 validator geth[316890]: INFO [08-22|15:18:16.499] Loaded most recent local header          number=15,390,970 hash=aac1cd..1d3872 td=56,955,543,553,507,183,816,410 age=6m36s
Aug 22 15:18:16 validator geth[316890]: INFO [08-22|15:18:16.500] Loaded most recent local full block      number=15,390,970 hash=aac1cd..1d3872 td=56,955,543,553,507,183,816,410 age=6m36s
Aug 22 15:18:16 validator geth[316890]: INFO [08-22|15:18:16.500] Loaded most recent local fast block      number=15,390,970 hash=aac1cd..1d3872 td=56,955,543,553,507,183,816,410 age=6m36s
Aug 22 15:18:16 validator geth[316890]: INFO [08-22|15:18:16.501] Loaded last fast-sync pivot marker       number=12,770,876
Aug 22 15:18:16 validator geth[316890]: WARN [08-22|15:18:16.509] Head state missing, repairing            number=15,390,970 hash=aac1cd..1d3872 snaproot=790081..9aed65

@yorickdowne
Copy link
Author

Yeah it's possible 3 minutes is not enough on RPi4. Just give it however much time it needs for a clean shutdown.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment