Pruning Geth 1.10.x

GethBEHAVE.md

Overview

Geth (Go-Ethereum) as of July 2022 takes about 650 GiB of space on a fast/snap sync, and then grows by ~ 14 GiB/week with default cache, ~ 8 GiB/week with more cache.

This will fill a 1TB SSD in a few months, to the point where space usage should be brought down again with an offline prune.

There is an odd behavior with Geth 1.10.20 where initial DB size is large after sync until Geth has been restarted twice. This is expected to be fixed soon, possibly as soon as 1.10.21.

Happily, Geth 1.10.x introduces "snapshot offline prune", which brings it back down to about its original size. It takes roughly 4-6 hours to prune the Geth database, and this has to be done while Geth is not running.

Caveat that while several folx have used offline pruning successfully, there is risk associated with it. The two failure modes we have seen already are:

• There is 25 GiB or less of free disk space
• The pruning process is interrupted partway through

Geth is working on a state storage change that would drastically reduce its on-disk growth, in a future version of Geth. Do keep an eye on that.

Prerequisites

• This is not an archive node. Do not try to prune an archive node.
• The volume Geth stores its database on has roughly 40 GiB of free space or more. We know 25 GiB is not enough.
• Geth 1.10.x installed
• Geth is fully synced
• Geth has finished creating a snapshot, and this snapshot is 128 blocks old or older, about 35 minutes. You can tell it is done creating the snapshot when it is no longer showing "generating snapshot" messages in logs. Geth generates a snapshot by default, right after it is done syncing.
• tmux or similar such as screen installed: sudo apt install tmux. This intro is useful for navigating tmux. tmux just makes sure the prune process continues even when the user is logged out, e.g. because of an idle timer or connection break.

What you expect to see

Geth will prune in 3 stages: "Iterating state snapshot", "Pruning state data", and "Compacting database". During the "Compacting database" stage, it may not output any log entries for an hour or so (mainstream SSD IOPS). Don't restart it when this happens, let it run!

If you see messages about "generating snapshot" and an ETA during the prune, you don't actually have a snapshot yet! Either the --datadir and/or USER aren't right, or Geth just didn't have enough time to complete the snapshot. In that case, do stop the process, run Geth normally again, and observe its logs until snapshot has completed and is 128 blocks old.

When Geth is done pruning, the process will exit and you will see a log line that contains the phrase State pruning successful.

Pruning if you are using systemd to run Geth

systemd will run something like a geth service, with a User specified in the /etc/systemd/system/geth.service file, and an ExecStart in the same file that runs geth, which also specifies the --datadir path.

Stop Geth: sudo systemctl stop geth

You now have two options, choose whichever is easiest for you.

Systemd option A, use sudo

• First, start tmux or screen. This is so you can get disconnected and the prune will continue running.
• Then, with the USER and PATH to --datadir from the systemd service file, run sudo -u USER geth --datadir PATH snapshot prune-state. If you set up Geth following Somer Esat's guide, that's sudo -u goeth geth --datadir /var/lib/goethereum snapshot prune-state. If you followed CoinCashew's instructions to set up Geth, it'd just be geth snapshot prune-state.

Note that running as the user Geth usually runs as is critical for the Geth service to still have permissions to its own DB files, when you start it up again.

Once pruning is complete, start Geth again: sudo systemctl start geth

If you don't want to run tmux, you could modify the Geth service instead.

Systemd option B, modify the existing service

• Edit the existing file: sudo nano /etc/systemd/system/geth.service and add this to the very end of ExecStart: snapshot prune-state

Add this to the existing arguments, do not replace the existing arguments. Geth still needs to know where its --datadir is at.

• Tell systemd you made changes: sudo systemctl daemon-reload
• Start the Geth service: sudo systemctl start geth
• You can observe prune progress with journalctl -fu geth

Note: Unless you also change the restart parameter, systemd will restart the prune after it finishes, which will fail. Once you restore the service to its previous state, you expect Geth to run successfully again.

Once Geth has finished pruning, undo the changes you made:

• Edit the existing file: sudo nano /etc/systemd/system/geth.service and remove this from ExecStart: snapshot prune-state
• Tell systemd you made changes: sudo systemctl daemon-reload
• Start the Geth service: sudo systemctl start geth
• You can observe that Geth starts correctly with journalctl -fu geth

Pruning if you are using docker-compose to run Geth

If you are using docker-compose, all you need to do is stop the Geth service, and start it again with pruning parameters. This has been tested with eth-docker.

eth-docker supports ./ethd prune-geth which handles the below steps for you. It also offers an auto-prune.sh script that can kick off pruning when disk space goes below a threshold, or will just output a warning that crontab can email to you if run as auto-prune.sh --dry-run.

Rocketpool uses rocketpool service prune-eth1 to prune Geth

• docker-compose stop execution && docker-compose rm execution
• docker-compose run --rm --name geth_prune -d execution snapshot prune-state
• Observe pruning progress with: docker logs -f --tail 500 geth_prune
• When pruning is done: docker-compose up -d execution
• And observe that Geth is running correctly: docker-compose logs -f execution