yorickdowne/GethBEHAVE.md

## GethBEHAVE.md

      
    Raw
  

              GethBEHAVE.md
            
          
Note: PBSS in Geth >=1.13.0 removes the need to prune manually.


Old content for reference

Overview

Geth (Go-Ethereum) as of July 2022 takes about 650 GiB of space on a fast/snap sync, and then grows by ~ 14 GiB/week with default cache, ~ 8 GiB/week with more cache.
This will fill a 2TB SSD in a year or two, to the point where space usage should be brought down again with an offline prune.
Happily, Geth 1.10.x introduced "snapshot offline prune", which brings it back down to about its original size. It takes roughly 4-6 hours to prune the Geth database, and this has to be done while Geth is not running.
Caveat that while several folx have used offline pruning successfully, there is risk associated with it. The two failure modes
we have seen already are:

There is 37 GiB or less of free disk space
The pruning process is interrupted partway through.

Prerequisites


 This is not an archive node. Do not try to prune an archive node.
 The volume Geth stores its database on has roughly 40 (?) GiB of free space or more. We know 37 GiB is not enough.
 Geth is fully synced
 Geth has finished creating a snapshot, and this snapshot is 128 blocks old or older, about 35 minutes. You can tell it is done creating the snapshot when it is no longer showing "state snapshot generation" messages in logs. Geth generates a snapshot by default, right after it is done syncing.
 tmux or similar such as screen installed: sudo apt install tmux. This intro is useful for navigating tmux. tmux just makes sure the prune process continues even when the user is logged out, e.g. because of an idle timer or connection break.

What you expect to see

Geth will prune in 3 stages: "Iterating state snapshot", "Pruning state data", and "Compacting database". During the "Compacting database" stage, it may not output any log entries for an hour or so (mainstream SSD IOPS). Don't restart it when this happens, let it run!
If you see messages about "state snapshot generation" during the prune, you don't actually have a snapshot yet! Either the --datadir and/or USER aren't right, or Geth just didn't have enough time to complete the snapshot. In that case, do stop the process, run Geth normally again, and observe its logs until snapshot has completed and is 128 blocks old.
When Geth is done pruning, the process will exit and you will see a log line that contains the phrase State pruning successful.
Pruning if you are using systemd to run Geth

systemd will run something like a geth service, with a User specified in the /etc/systemd/system/geth.service file,
and an ExecStart in the same file that runs geth, which also specifies the --datadir path.
Stop Geth: sudo systemctl stop geth

If Geth does not have enough time to shut down cleanly, the prune may fail. You may need to give it 180s to shut down cleanly, depending on your storage hardware

You now have two options, choose whichever is easiest for you.
Systemd option A, use sudo


First, start tmux or screen. This is so you can get disconnected and the prune will continue running.
Then, with the USER and PATH to --datadir from the systemd service file, run sudo -u USER geth --datadir PATH snapshot prune-state. If you set up Geth following Somer Esat's current guide, that's sudo -u geth geth --datadir /var/lib/geth snapshot prune-state, or with Somer's original guide it's sudo -u goeth geth --datadir /var/lib/goethereum snapshot prune-state. If you followed CoinCashew's instructions to set up Geth, it'd just be geth snapshot prune-state.

Note that running as the user Geth usually runs as is critical for the Geth service to still have permissions to its own DB files, when
you start it up again.
Once pruning is complete, start Geth again: sudo systemctl start geth
If you don't want to run tmux, you could modify the Geth service instead.
Systemd option B, modify the existing service


Edit the existing file: sudo nano /etc/systemd/system/geth.service and add this to the very end of ExecStart: snapshot prune-state


Add this to the existing arguments, do not replace the existing arguments. Geth still needs to know where its --datadir is at.


Tell systemd you made changes: sudo systemctl daemon-reload
Start the Geth service: sudo systemctl start geth
You can observe prune progress with journalctl -fu geth

Note: Unless you also change the restart parameter, systemd will restart the prune after it finishes, which will fail. Once you restore the service to its previous state, you expect Geth to run successfully again.
Once Geth has finished pruning, undo the changes you made:

Edit the existing file: sudo nano /etc/systemd/system/geth.service and remove this from ExecStart: snapshot prune-state
Tell systemd you made changes: sudo systemctl daemon-reload
Start the Geth service: sudo systemctl start geth
You can observe that Geth starts correctly with journalctl -fu geth

Pruning if you are using docker-compose to run Geth

If you are using docker-compose, all you need to do is stop the Geth service, and start it again with pruning parameters.

eth-docker supports ./ethd prune-geth which handles the below steps for you. It also offers an auto-prune.sh script
that can kick off pruning when disk space goes below a threshold, or will just output a warning that crontab can email to you
if run as auto-prune.sh --dry-run.


Rocketpool uses rocketpool service prune-eth1 to prune Geth

Generic docker-compose, this won't work in eth-docker:

docker-compose stop execution && docker-compose rm execution
docker-compose run --rm --name geth_prune -d execution snapshot prune-state
Observe pruning progress with: docker logs -f --tail 500 geth_prune
When pruning is done: docker-compose up -d execution
And observe that Geth is running correctly: docker-compose logs -f execution