Skip to content

Instantly share code, notes, and snippets.

@86b

86b/ethereum-setup-guide-blocknet.md Secret

Created January 15, 2020 09:33
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save 86b/0336522778cb0e89fbb769392287f721 to your computer and use it in GitHub Desktop.
Save 86b/0336522778cb0e89fbb769392287f721 to your computer and use it in GitHub Desktop.
Download ZIP
Ethereum Setup Guide - Blocknet
Raw
ethereum-setup-guide-blocknet.md

Ethereum Setup Guide for Blocknet Service Nodes

The following guide will help get Blocknet Service Nodes staged to support decentralized Ethereum services on the Blocknet Protocol. We recommend staging in Docker containers, but users can leverage whatever environment they're most comfortable with in order to get a full ETH archival node in sync.

In this guide we'll cover getting both Parity and Go-Ethereum (geth) full nodes in sync - we don't have one in particular that we prefer, so feel free to use whichever one you like most and are most comfortable with. We'll cover getting both Parity and geth setup in Docker as well as built from source and setup using an install script. These guides will be based on Ubuntu 18.04.

Things to note

  • It is recommended to run ETH on a Linux-based machine or VM. For example, Parity is single-threaded when run on Windows! We've also seen issues related to port consumption on Windows (WSL included) which lead to all ports being consumed causing a mess!

  • SSDs make a huge difference! There are hundreds of transactions each Ethereum block - you will likely never get a full archival node synced up with a HDD.

  • The Ethereum blockchain is quite large and currently is about 3.6 TB in size! Please plan accordingly as the blockchain's size is increasing by about 1.6 TB/year!

  • Although we've found Ethereum to be extremely durable (has a fair amount of resistance to ungraceful shutdowns and such), you're going to want to make sure you have regular backups just in case something disastrous happens that corrupts your block database.

  • Patience is required! On a decent machine it can still take over 2 months to get Ethereum synced.

geth Docker Setup Guide

If you don’t have Docker installed, there are various guides you can follow including this one here.

Staging

For an easy way to persist your blockchain data outside of your Docker container on your host machine, you should leverage the docker -v functionality for your geth container.

Step 1

Choose a location you plan on using for your geth volume. For this example we'll use /opt/geth/.ethereum

Step 2

Create the directory and enter it

  • mkdir -p /opt/geth/.ethereum
  • cd /opt/geth/.ethereum

Step 3

Create your geth configuration file config.toml and paste the following in it and save the file:

[Eth]
NetworkId = 1
SyncMode = "full"
NoPruning = true
NoPrefetch = false
UltraLightFraction = 75
DatabaseCache = 3072
DatabaseFreezer = ""
TrieCleanCache = 1536
TrieDirtyCache = 1536
TrieTimeout = 3600000000000
EnablePreimageRecording = false
EWASMInterpreter = ""
EVMInterpreter = ""

[Eth.Miner]
GasFloor = 8000000
GasCeil = 8000000
GasPrice = 1000000000
Recommit = 3000000000
Noverify = false

[Eth.Ethash]
CacheDir = "ethash"
CachesInMem = 2
CachesOnDisk = 3
DatasetDir = "/root/.ethash"
DatasetsInMem = 1
DatasetsOnDisk = 2
PowMode = 0

[Eth.TxPool]
Locals = []
NoLocals = false
Journal = "transactions.rlp"
Rejournal = 3600000000000
PriceLimit = 1
PriceBump = 10
AccountSlots = 16
GlobalSlots = 4096
AccountQueue = 64
GlobalQueue = 1024
Lifetime = 10800000000000

[Eth.GPO]
Blocks = 20
Percentile = 60

[Shh]
MaxMessageSize = 1048576
MinimumAcceptedPOW = 2e-01
RestrictConnectionBetweenLightClients = true

[Node]
DataDir = "/root/.ethereum"
omitempty = ""
NoUSB = true
IPCPath = "geth.ipc"
HTTPPort = 8545
HTTPVirtualHosts = ["localhost"]
HTTPModules = ["net", "web3", "eth", "shh"]
WSPort = 8546
WSModules = ["net", "web3", "eth", "shh"]
GraphQLPort = 8547
GraphQLVirtualHosts = ["localhost"]

[Node.P2P]
MaxPeers = 250
MaxPendingPeers = 50
NoDiscovery = false
BootstrapNodes = ["enode://d860a01f9722d78051619d1e2351aba3f43f943f6f00718d1b9baa4101932a1f5011f16bb2b1bb35db20d6fe28fa0bf09636d26a87d31de9ec6203eeedb1f666@18.138.108.67:30303", "enode://22a8232c3abc76a16ae9d6c3b164f98775fe226f0917b0ca871128a74a8e9630b458460865bab457221f1d448dd9791d24c4e5d88786180ac185df813a68d4de@3.209.45.79:30303", "enode://ca6de62fce278f96aea6ec5a2daadb877e51651247cb96ee310a318def462913b653963c155a0ef6c7d50048bba6e6cea881130857413d9f50a621546b590758@34.255.23.113:30303", "enode://279944d8dcd428dffaa7436f25ca0ca43ae19e7bcf94a8fb7d1641651f92d121e972ac2e8f381414b80cc8e5555811c2ec6e1a99bb009b3f53c4c69923e11bd8@35.158.244.151:30303", "enode://8499da03c47d637b20eee24eec3c356c9a2e6148d6fe25ca195c7949ab8ec2c03e3556126b0d7ed644675e78c4318b08691b7b57de10e5f0d40d05b09238fa0a@52.187.207.27:30303", "enode://103858bdb88756c71f15e9b5e09b56dc1be52f0a5021d46301dbbfb7e130029cc9d0d6f73f693bc29b665770fff7da4d34f3c6379fe12721b5d7a0bcb5ca1fc1@191.234.162.198:30303", "enode://715171f50508aba88aecd1250af392a45a330af91d7b90701c436b618c86aaa1589c9184561907bebbb56439b8f8787bc01f49a7c77276c58c1b09822d75e8e8@52.231.165.108:30303", "enode://5d6d7cd20d6da4bb83a1d28cadb5d409b64edf314c0335df658c1a54e32c7c4a7ab7823d57c39b6a757556e68ff1df17c748b698544a55cb488b52479a92b60f@104.42.217.25:30303", "enode://979b7fa28feeb35a4741660a16076f1943202cb72b6af70d327f053e248bab9ba81760f39d0701ef1d8f89cc1fbd2cacba0710a12cd5314d5e0c9021aa3637f9@5.1.83.226:30303"]
BootstrapNodesV5 = ["enode://06051a5573c81934c9554ef2898eb13b33a34b94cf36b202b69fde139ca17a85051979867720d4bdae4323d4943ddf9aeeb6643633aa656e0be843659795007a@35.177.226.168:30303", "enode://0cc5f5ffb5d9098c8b8c62325f3797f56509bff942704687b6530992ac706e2cb946b90a34f1f19548cd3c7baccbcaea354531e5983c7d1bc0dee16ce4b6440b@40.118.3.223:30304", "enode://1c7a64d76c0334b0418c004af2f67c50e36a3be60b5e4790bdac0439d21603469a85fad36f2473c9a80eb043ae60936df905fa28f1ff614c3e5dc34f15dcd2dc@40.118.3.223:30306", "enode://85c85d7143ae8bb96924f2b54f1b3e70d8c4d367af305325d30a61385a432f247d2c75c45c6b4a60335060d072d7f5b35dd1d4c45f76941f62a4f83b6e75daaf@40.118.3.223:30307"]
StaticNodes = []
TrustedNodes = []
ListenAddr = ":30303"
EnableMsgEvents = false

[Node.HTTPTimeouts]
ReadTimeout = 30000000000
WriteTimeout = 30000000000
IdleTimeout = 120000000000

Done!

Setup

Now that you're properly staged, we can pull the image and run the geth Docker container.

Step 1

Pull the official geth Docker Image:

  • docker pull ethereum/client-go

Step 2

Run your container:

  • docker run -d -p 8545:8545 -p 8546:8546 -p 8547:8547 -p 30303:30303 -v /opt/geth/.ethereum:/root/.ethereum --name geth ethereum/client-go --config /root/.ethereum/config.toml

Done!

if you run into permission denied issues related to leveraging docker -v, please research chown such as chown -R 1000:1000 /opt/geth

Controlling Your geth Container

The docker run command from the Setup section is used to first run and start your container. After this, you'll be leveraging the docker attach, docker stop, and docker start commands.

To gracefully shutdown geth running in your container, follow these steps:

Step 1

Enter your container

  • docker attach geth

Step 2

Gracefully shutdown geth

  • ctrl+c or
  • pkill -SIGINT geth

Done!

After geth shuts down, you should automatically exit the container - if for some reason you do not, make sure geth is indeed shutdown by checking top. When you confirm geth is shut down, you can then proceed to type and enter exit to exit the container and subsequently stop it with docker stop geth.

Warning: Stopping a container with geth actively running in it can lead to database corruption issues!

Starting, Entering, and Exiting the Container

  • To start the container: docker start geth

  • To enter the container: docker attach geth

  • To exit the container and keep it running: ctrl+p+q

Removing the Container

  • With the container stopped, do docker rm geth

Reading your geth Container Logs

From outside the container:

  • docker logs -f geth where -f will follow the console output

If you have quite a large output because geth has been running for a while, you can choose to only check back a few lines from the current spot by:

  • docker logs -f --tail 10 geth where 10 is how many lines you wish to display backwards from the current spot.

If you wish to look for something specific (for example Imported), you can leverage grep:

  • docker logs -f --tail 10 geth | grep Imported

  • To exit the logs, simply ctrl+c

Enter the container in a system shell

  • docker exec -ti geth /bin/sh

To exit:

  • exit or ctrl+p+q

Building geth From Source

The following script will install all required dependencies, build and install geth, and generate the required config files.

Step 1

Create geth.sh and paste the following contents into the file and save it:

# geth install script

apt update \
  && apt install -y curl vim wget git \
  && apt install -y \
    gcc g++ pkg-config file make cmake build-essential software-properties-common

add-apt-repository -y ppa:longsleep/golang-backports

apt update \
  && apt install -y golang-go

git clone https://github.com/ethereum/go-ethereum \
  && cd go-ethereum

make geth

mkdir ~/.ethereum \
  && cd ~/.ethereum

cat > config.toml << EOF
[Eth]
NetworkId = 1
SyncMode = "full"
NoPruning = true
NoPrefetch = false
UltraLightFraction = 75
DatabaseCache = 3072
DatabaseFreezer = ""
TrieCleanCache = 1536
TrieDirtyCache = 1536
TrieTimeout = 3600000000000
EnablePreimageRecording = false
EWASMInterpreter = ""
EVMInterpreter = ""
[Eth.Miner]
GasFloor = 8000000
GasCeil = 8000000
GasPrice = 1000000000
Recommit = 3000000000
Noverify = false
[Eth.Ethash]
CacheDir = "ethash"
CachesInMem = 2
CachesOnDisk = 3
DatasetDir = "/root/.ethash"
DatasetsInMem = 1
DatasetsOnDisk = 2
PowMode = 0
[Eth.TxPool]
Locals = []
NoLocals = false
Journal = "transactions.rlp"
Rejournal = 3600000000000
PriceLimit = 1
PriceBump = 10
AccountSlots = 16
GlobalSlots = 4096
AccountQueue = 64
GlobalQueue = 1024
Lifetime = 10800000000000
[Eth.GPO]
Blocks = 20
Percentile = 60
[Shh]
MaxMessageSize = 1048576
MinimumAcceptedPOW = 2e-01
RestrictConnectionBetweenLightClients = true
[Node]
DataDir = "/root/.ethereum"
omitempty = ""
NoUSB = true
IPCPath = "geth.ipc"
HTTPPort = 8545
HTTPVirtualHosts = ["localhost"]
HTTPModules = ["net", "web3", "eth", "shh"]
WSPort = 8546
WSModules = ["net", "web3", "eth", "shh"]
GraphQLPort = 8547
GraphQLVirtualHosts = ["localhost"]
[Node.P2P]
MaxPeers = 250
MaxPendingPeers = 50
NoDiscovery = false
BootstrapNodes = ["enode://d860a01f9722d78051619d1e2351aba3f43f943f6f00718d1b9baa4101932a1f5011f16bb2b1bb35db20d6fe28fa0bf09636d26a87d31de9ec6203eeedb1f666@18.138.108.67:30303", "enode://22a8232c3abc76a16ae9d6c3b164f98775fe226f0917b0ca871128a74a8e9630b458460865bab457221f1d448dd9791d24c4e5d88786180ac185df813a68d4de@3.209.45.79:30303", "enode://ca6de62fce278f96aea6ec5a2daadb877e51651247cb96ee310a318def462913b653963c155a0ef6c7d50048bba6e6cea881130857413d9f50a621546b590758@34.255.23.113:30303", "enode://279944d8dcd428dffaa7436f25ca0ca43ae19e7bcf94a8fb7d1641651f92d121e972ac2e8f381414b80cc8e5555811c2ec6e1a99bb009b3f53c4c69923e11bd8@35.158.244.151:30303", "enode://8499da03c47d637b20eee24eec3c356c9a2e6148d6fe25ca195c7949ab8ec2c03e3556126b0d7ed644675e78c4318b08691b7b57de10e5f0d40d05b09238fa0a@52.187.207.27:30303", "enode://103858bdb88756c71f15e9b5e09b56dc1be52f0a5021d46301dbbfb7e130029cc9d0d6f73f693bc29b665770fff7da4d34f3c6379fe12721b5d7a0bcb5ca1fc1@191.234.162.198:30303", "enode://715171f50508aba88aecd1250af392a45a330af91d7b90701c436b618c86aaa1589c9184561907bebbb56439b8f8787bc01f49a7c77276c58c1b09822d75e8e8@52.231.165.108:30303", "enode://5d6d7cd20d6da4bb83a1d28cadb5d409b64edf314c0335df658c1a54e32c7c4a7ab7823d57c39b6a757556e68ff1df17c748b698544a55cb488b52479a92b60f@104.42.217.25:30303", "enode://979b7fa28feeb35a4741660a16076f1943202cb72b6af70d327f053e248bab9ba81760f39d0701ef1d8f89cc1fbd2cacba0710a12cd5314d5e0c9021aa3637f9@5.1.83.226:30303"]
BootstrapNodesV5 = ["enode://06051a5573c81934c9554ef2898eb13b33a34b94cf36b202b69fde139ca17a85051979867720d4bdae4323d4943ddf9aeeb6643633aa656e0be843659795007a@35.177.226.168:30303", "enode://0cc5f5ffb5d9098c8b8c62325f3797f56509bff942704687b6530992ac706e2cb946b90a34f1f19548cd3c7baccbcaea354531e5983c7d1bc0dee16ce4b6440b@40.118.3.223:30304", "enode://1c7a64d76c0334b0418c004af2f67c50e36a3be60b5e4790bdac0439d21603469a85fad36f2473c9a80eb043ae60936df905fa28f1ff614c3e5dc34f15dcd2dc@40.118.3.223:30306", "enode://85c85d7143ae8bb96924f2b54f1b3e70d8c4d367af305325d30a61385a432f247d2c75c45c6b4a60335060d072d7f5b35dd1d4c45f76941f62a4f83b6e75daaf@40.118.3.223:30307"]
StaticNodes = []
TrustedNodes = []
ListenAddr = ":30303"
EnableMsgEvents = false
[Node.HTTPTimeouts]
ReadTimeout = 30000000000
WriteTimeout = 30000000000
IdleTimeout = 120000000000
EOF

cd ~

Step 2

Run the script

  • chmod +x geth.sh

  • ./geth.sh

Done!

Starting geth

After the install script is finished, you can now navigate to and start geth:

  • cd ~/go-ethereum/build/bin

  • ./geth --config ~/.ethereum/config.toml

Stopping geth

To stop geth, you can ctrl+c the active console or you can pkill --signal SIGINT geth until you see it shut down and subsequently disappear in top

parity Docker Setup Guide

If you don’t have Docker installed, there are various guides you can follow including this one here.

Staging

For an easy way to persist your blockchain data outside of your Docker container to your host machine and desired location, you should leverage the docker -v functionality for your parity container.

Step 1

Choose a location you plan on using for your parity volume. For this example we'll use /opt/parity

Step 2

Create the directory and enter it

  • mkdir -p /opt/parity
  • cd /opt/parity

Step 3

Create your parity configuration file named config.toml and paste the following in it and save it:

[network]
warp = false
max_peers = 250
max_pending_peers = 75
no_serve_light = true
discovery = true
port = 30303

[rpc]
# Threads for handling incoming connections for HTTP JSON-RPC server.
# server_threads = 4
# Turn on additional processing threads for JSON-RPC servers (all transports). Setting this to a non-zero value allows parallel execution of cpu-heavy queries.
# processing_threads = 4
disable = false

[footprint]
# Compute and Store tracing data. (Enables trace_* APIs).
tracing = "on"
# Keep all state trie data. No pruning.
pruning = "archive"
# Enables Fat DB
fat_db = "on"
db_compaction = "ssd"
cache_size = 6144
scale_verifiers = true

[snapshots]
disable_periodic = true

[websockets]
disable = true

[ipc]
disable = true

Done!

if you run into permission denied issues related to leveraging docker -v, please research chown such as chown -R 1000:1000 /opt/parity

Setup

Now that you're properly staged, we can pull the image and run the parity Docker container.

Step 1

Pull the official parity Docker Image:

  • docker pull parity/parity

Step 2

Run your container:

  • docker run -d -p 5001:5001 -p 30303:30303 -v /opt/parity:/root/.local/share/io.parity.ethereum --name parity parity/parity

Done!

if you run into permission denied issues related to leveraging docker -v, please research chown such as chown -R 1000:1000 /opt/parity

Controlling Your parity Container

The docker run command from the Setup section is used to first run and start your container. After this, you'll be leveraging the docker attach, docker stop, and docker start commands.

To gracefully shutdown parity running in your container, follow these steps:

Step 1

Enter your container

  • docker attach parity

Step 2

Gracefully shutdown parity

  • ctrl+c or
  • pkill --signal SIGINT parity

Another trick you can use to gracefully shutdown your parity container is:

  • docker kill --signal=SIGINT parity from outside the container

Done!

After parity shuts down, you should automatically exit the container - if for some reason you do not, make sure parity is indeed shutdown by checking top. When you confirm parity is shut down, you can then proceed to type and enter exit to exit the container and subsequently stop it with docker stop parity.

Warning: Stopping a container with parity actively running in it can lead to database corruption issues!

Starting, Entering, and Exiting the Container

  • To start the container: docker start parity

  • To enter the container: docker attach parity

  • To exit the container and keep it running: ctrl+p+q

Removing the Container

  • With the container stopped, docker rm parity

Reading your parity Container Logs

From outside the container:

  • docker logs -f parity - where -f will follow the console output

If you have quite a large output because parity has been running for a while, you can choose to only check back a few lines from the current spot by:

  • docker logs -f --tail 10 parity where 10 is how many lines you wish to display backwards from the current spot

If you wish to look for something specific (for example, Imported), you can leverage grep:

  • docker logs -f --tail 10 parity 2>&1 | grep Imported

Enter the container in a bash shell

  • docker exec -ti parity /bin/bash

To exit:

  • exit or ctrl+p+q

Building parity From Source

The following script will install all required dependencies, build and install parity, and generate the required config files.

Step 1

Create parity.sh and paste the following contents into the file and save it:

# Parity install
cd ~

apt update \
  && apt install -y curl vim wget git \
  && apt install -y \
    gcc g++ pkg-config file make cmake llvm-7-dev clang cargo \
    libudev-dev perl yasm

curl https://sh.rustup.rs -sSf | sh && 1

git clone https://github.com/paritytech/parity-ethereum

cd parity-ethereum

git checkout stable

cargo build --release --features final

mkdir -p ~/.local/share/io.parity.ethereum
  && cd ~/.local/share/io.parity.ethereum

cat > config.toml << EOF
# This config should be placed in following path:
#   ~/.local/share/io.parity.ethereum/config.toml
[network]
warp = false
max_peers = 250
max_pending_peers = 75
no_serve_light = true
discovery = true
port = 30303
[rpc]
# Threads for handling incoming connections for HTTP JSON-RPC server.
# server_threads = 4
# Turn on additional processing threads for JSON-RPC servers (all transports). Setting this to a non-zero value allows parallel execution of cpu-heavy queries.
# processing_threads = 4
disable = false
[footprint]
# Compute and Store tracing data. (Enables trace_* APIs).
tracing = "on"
# Keep all state trie data. No pruning.
pruning = "archive"
# Enables Fat DB
fat_db = "on"
db_compaction = "ssd"
cache_size = 6144
scale_verifiers = true
[snapshots]
disable_periodic = true
[websockets]
disable = true
[ipc]
disable = true
EOF

cd ~

Step 2

Run the script

  • chmod +x parity.sh

  • ./parity.sh

Done!

Starting parity

After the install script is finished, you can now navigate to and start parity:

  • cd ~/parity-ethereum/target/release

  • ./parity

Stopping parity

To stop parity, you can ctrl+c the active console or you can pkill --signal SIGINT parity until you see it shut down and subsequently disappear in top

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment