In this procedure, we are attempting to set up a devnet, which is a blockchain completely local to your machine. Below is the attempted procedure to get it up and running.
In general, believe this is the structure we want:
graph TB;
reth1[reth execution client]
lighthouse1[Lighthouse consensus client]
reth2[reth execution client]
lighthouse2[Lighthouse consensus client]
reth1 <--> reth2
lighthouse1 <--> lighthouse2
subgraph node1 [Node 1]
direction LR
lighthouse1 <--> reth1
end
subgraph node2 [Node 2]
direction LR
lighthouse2 <--> reth2
end
The files in this folder are mostly copies of
https://github.com/ethpandaops/withdrawals-testnet/tree/master/zhejiang-testnet/custom_config_data,
but the chain ID is changed to 123456.
All commands are run from this directory.
-
Generate two database files for each node (since each one locks the database they need to have separate ones). Use the
genesis.jsonfile at the bottom of the wiki:reth init --chain genesis.json --db node1 reth init --chain genesis.json --db node2 -
Enable
127.0.0.2as a loopback device so that more that one address can serve from localhost.sudo ifconfig lo0 alias 127.0.0.2 -
Run the first node using the following commands. Note I am unsure if the
debug.tipis needed, but it warns you if you don't add it. If you are using the above genesis file, this tip should bed8f5a1b47c842e0e14c62205adaf926e4f1df17c6245fd1e7fbc6a41375eb72e
RUST_LOG=info,sync::stages=trace,downloaders=trace,debug reth node \
--chain genesis.json \
--db node1 \
-vvvvv \ # Print out all the info for now
-d \ # Disable peer discovery since the nodes would otherwise conflict for a port.
--http.addr 127.0.0.1 \ # All set to 127.0.0.1 for consistency.
--ws.addr 127.0.0.1. \
--authrpc.addr 127.0.0.1 \
--discovery.port 30303 \ # This is the default port, just specified explicitly
--port 30303 \
--http \ # enable http rpc server
--debug.tip d8f5a1b47c842e0e14c62205adaf926e4f1df17c6245fd1e7fbc6a41375eb72e
You should get output that looks like this:
2023-03-22T22:20:58.633015Z INFO Loading saved peers file=/Users/ryan/Library/Application Support/reth/net/known-peers.json
2023-03-22T22:20:58.642174Z INFO Connected to P2P network peer_id=0x0dd7bdceac2862305129078e70a317d89e260d73d8ee7d316c7938e0ae85f3ebeaf96da259ec5c9c37219c694ac47de516c2ba70f652c291ac14b0c02017c834 local_addr=0.0.0.0:30303
2023-03-22T22:20:58.642187Z INFO Test transaction pool initialized
The peer ID you will need to connect to this node is 0dd7bdceac2862305129078e70a317d89e260d73d8ee7d316c7938e0ae85f3ebeaf96da259ec5c9c37219c694ac47de516c2ba70f652c291ac14b0c02017c834. Call this $NODE1_ID
IMPORTANT: this will generate a jwt.hex file in the reth configuration directory (see notes below). Copy this file to another location so that you can use it to run a consensus client on top of the first node. On a Mac you can do this
cp ~/Library/Application\ Support/reth/jwtsecret/jwt.hex jwt1.hex- Start the second node:
RUST_LOG=info,sync::stages=trace,downloaders=trace,debug reth node \
--chain genesis.json \
--db node2 \
-vvvvv \
-d \
--http.addr 127.0.0.2 \ # All set to 127.0.0.2 for consistency. Note the different IP address here
--ws.addr 127.0.0.2 \
--authrpc.addr 127.0.0.2 \
--discovery.port 30304 \ # note the different discovery port from above to prevent an os open error.
--port 30304 \
--http \ # enable http rpc server
--trusted-peers enode://$NODE1_ID@127.0.0.1:30303 \ # Info on the first node to pair to.
--debug.tip d8f5a1b47c842e0e14c62205adaf926e4f1df17c6245fd1e7fbc6a41375eb72e
If it works, the first node should print out:
2023-03-22T22:26:03.774975Z INFO Session established remote_addr=127.0.0.1:56095 peer_id=0x2fb7d090fab1dbd96e56aa9243845073c7fea44a1d5c6712bf8ee3f93abfc14f4218981112cd4fa66ba619c00f2e64effd29cedd9600147c11fe82089221f3a5 total_active=1
2023-03-22T22:26:03.775033Z INFO Peer connected connected_peers=1 peer_id=0x2fb7…f3a5 best_block=0xd8f5…b72e
while the second node will print out
2023-03-22T22:26:03.774989Z INFO Session established remote_addr=127.0.0.1:30303 peer_id=0x0dd7bdceac2862305129078e70a317d89e260d73d8ee7d316c7938e0ae85f3ebeaf96da259ec5c9c37219c694ac47de516c2ba70f652c291ac14b0c02017c834 total_active=1
2023-03-22T22:26:03.775034Z INFO Peer connected connected_peers=1 peer_id=0x0dd7…c834 best_block=0xd8f5…b72e
Currently it does nothing after peering, and at some point one peer will send a disconnect request to the other.
2023-03-22T17:46:37.168583Z WARN Peer disconnected. connected_peers=0 peer_id=0xbb76…9c81 reason=None
2023-03-22T17:46:38.166856Z INFO Session established remote_addr=127.0.0.1:30303 peer_id=0xbb76309b57c4bc20433a390511659870e0d532e6549d5caef19f9b17e0bead09208fa83a1f696138fecf5d55ae7cc5cc7e4b76a3f8e7cf64c56abe05bbef9c81 total_active=1
2023-03-22T17:46:38.166947Z INFO Peer connected connected_peers=1 peer_id=0xbb76…9c81 best_block=0x8275…97ec
2023-03-22T17:46:38.167189Z ERROR failed to receive message err=P2PStreamError(Disconnected(DisconnectRequested)) remote_peer_id=0xbb76309b57c4bc20433a390511659870e0d532e6549d5caef19f9b17e0bead09208fa83a1f696138fecf5d55ae7cc5cc7e4b76a3f8e7cf64c56abe05bbef9c81
2023-03-22T17:46:38.167361Z WARN Peer disconnected. connected_peers=0 peer_id=0xbb76…9c81 reason=Disconnect requested
2023-03-22T17:46:56.161540Z INFO Status connected_peers=0 stage=None checkpoint=0
IMPORTANT: this will generate a jwt.hex file in the reth configuration directory (see notes below), overwriting the jwt.hex from the first node. Copy this file to another location so that you can use it to run a consensus client on top of the second node. If you are on a mac you can do this
cp ~/Library/Application\ Support/reth/jwtsecret/jwt.hex jwt2.hex- You can attempt to connect lighthouse (a consensus client) to node 1 using the following
lighthouse \
--testnet-dir="." \
bn \
--datadir=lighthouse-datadir \
--eth1 \
--http \
--http-allow-sync-stalled \
--execution-endpoints=http://127.0.0.1:8551 \
--enr-udp-port=9000 \
--enr-tcp-port=9000 \
--discovery-port=9000 \
--jwt-secrets=jwt1.hexThis will connect lighthouse to node1 in some capacity, although the consensis client will continue repeating the message
Mar 22 22:30:53.213 ERRO Failed to update eth1 deposit cache error: GetDepositLogsFailed("eth_getLogs call failed ServerMessage { code: -32601, message: \"Method not found\" }"), service: beacon
Mar 22 22:30:53.214 INFO Waiting for more deposits valid_deposits: None, total_deposits: 0, min_genesis_active_validators: 58000, service: beacon-
???
-
Profit!
The above setup is doing two things:
- Atttempting to set up a complete node (with a consensus client and an execution client)
- Attempting to peer two execution clients so that the two complete nodes can peer later.
We need to investigate the following.
- Determine who to run
castagainst. I believe this is the reth execution client. - Get a single complete node working.
- Get multiple complete nodes talking to each other.
-
When a
rethnode starts, it generates ajwtsecret key that the consensus client needs to be able to pair. It is generated in one of the following locations:- Linux:
$XDG_DATA_HOME/reth/jwtsecret/jwt.hexor$HOME/.local/share/reth/jwtsecret/jwt.hex - Windows:
{FOLDERID_RoamingAppData}/reth/jwtsecret/jwt.hex - macOS:
$HOME/Library/Application Support/reth/jwtsecret/jwt.hex
This file is overwritten every time
reth nodeis run! So if you need thejwtsecret, copy it to another location. - Linux:
-
The RPC auth layer default port in
rethis8551, which matches the defaults expected by thelighthouseconsensus client. -
You can get even more verbose output from
rethby setting theRUST_LOGenvironmental variable. In their docs, they useRUST_LOG=info,sync::stages=trace,downloaders=trace
Note that this requires the following diff to
rethto enable the full peer ID to be printed: