UnOfficial Shelley Cconfig

SHELLEY_HOWTO.md

Introduction

This is a not official recommendations for Linux based pool nodes. This, is my node's config which runs on Ubuntu 18.04.3 LTS (Bionic Beaver) server (/w Jormungandr v0.8.4) and does not have any issues at all, since the changes.

Before the change of the settings to these below, my node was struggling and missed almost all of the leaders' time due to the forks that caused by not having the latest tip/blocks.

After, I changed the settings, I have not missed any schedule for creating blocks. The changes made on the last green vertical line (6pm):

HW/SW Requirements

Name	Min	Recommended
vCPU	2	4
RAM	2GB	4-8GB
Storage (SSD)	20GB SSD	40GB
Network IN	~2Mbps (1GB/hr i.e. )	5Mbps
Network OUT	~20Mbps (10GB/hr i.e. )	50Mbps
Limits (-n)	32768	65536or more
max_conn (Jormungandr)	4096	16384
log level (Jormungandr)	warn or off	warn or off

• Network latency depends on the location of the trusted peeers
  • Same region: 1-30ms
  • Other closer regions: 10-150ms
  • Other remote regions: 200-350ms
• Software Interrupts (for VMs), handle at least 2K

INT                NAME          RATE             MAX
56 [MSI 5767168-edge  ]   613 Ints/s     (max:   613)

• IOPs, not too much, 50 is more than enough /w 250kB/s writes and 100kB/s for reads

Device             tps    kB_read/s    kB_wrtn/s    kB_read    kB_wrtn
sda              22.63         7.46       243.60    1016667   33183380

Tweaking

TCP Stack tuning

Edit /etc/sysctl.conf

fs.file-max = 10000000
fs.nr_open = 10000000

net.core.netdev_max_backlog = 100000
net.core.somaxconn = 100000
net.ipv4.icmp_echo_ignore_broadcasts = 1
net.ipv4.ip_local_port_range = 1024 65535
net.ipv4.ip_nonlocal_bind = 1
net.ipv4.tcp_fin_timeout = 10
net.ipv4.tcp_keepalive_time = 1800
net.ipv4.tcp_keepalive_time = 300
net.ipv4.tcp_max_orphans = 262144
net.ipv4.tcp_max_syn_backlog = 100000
net.ipv4.tcp_max_tw_buckets = 262144
net.ipv4.tcp_mem = 786432 1697152 1945728
net.ipv4.tcp_reordering = 3
net.ipv4.tcp_rmem = 4096 87380 16777216
net.ipv4.tcp_sack = 0
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_syn_retries = 5
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_wmem = 4096 16384 16777216

net.netfilter.nf_conntrack_max = 10485760
net.netfilter.nf_conntrack_tcp_timeout_fin_wait = 30
net.netfilter.nf_conntrack_tcp_timeout_time_wait = 15

vm.swappiness = 10

Limits

Assumed that the fs.file-max and fs.file_nr from above have been already applied. Edit /etc/security/limits.conf

root               soft    nofile            32768
shelley            soft    nofile            32768
shelley            hard    nofile            1048576

If your service starts with systemctl then edit /etc/systemd/system/<YOURSTARTUPSCRIPT>.service file:

[Unit]
Description=Shelley Staking Pool
After=multi-user.target

[Service]
Type=simple
ExecStart=<YOUR_NODE_START_SCRIPT>


LimitNOFILE=16384 # Or more

Restart=on-failure
RestartSec=5s
User=shelley
Group=users

[Install]
WantedBy=multi-user.target

Jormungandr's config.yaml

  max_connections: 4096 # Recommended 16384 
  max_unreachable_nodes_to_connect_per_event: 128
  topics_of_interest:
    blocks: high
    messages: high

Logs

Disable intensive logging on production servers, so set the log level in the config.yaml file to either to "warn" or "off".

Trusted Peers

Sort them by your location, for fetching the initial Block0, and for bootstrapping.

    # IOHK US West - San francisco, California
    - address: "/ip4/52.9.132.248/tcp/3000"
      id: 671a9e7a5c739532668511bea823f0f5c5557c99b813456c
    # IOHK US West - San francisco, California
    - address: "/ip4/52.8.15.52/tcp/3000"
      id: 18bf81a75e5b15a49b843a66f61602e14d4261fb5595b5f5
    ## IOHK EU Central - Frankfurt, Germany
    - address: "/ip4/52.28.91.178/tcp/3000"
      id: 23b3ca09c644fe8098f64c24d75d9f79c8e058642e63a28c
    # IOHK EU Central - Frankfurt, Germany
    - address: "/ip4/3.125.75.156/tcp/3000"
      id: 22fb117f9f72f38b21bca5c0f069766c0d4327925d967791
    ## IOHK AP North East - Tokyo, Japan
    - address: "/ip4/13.114.196.228/tcp/3000"
      id: 7e1020c2e2107a849a8353876d047085f475c9bc646e42e9
    # IOHK AP North East - Tokyo, Japan
    - address: "/ip4/13.112.181.42/tcp/3000"
      id: 52762c49a84699d43c96fdfe6de18079fb2512077d6aa5bc
    # IOHK EU Central - Frankfurt, Germany
    - address: "/ip4/3.124.116.145/tcp/3000"
      id: 99cb10f53185fbef110472d45a36082905ee12df8a049b74

Access & Firewall

These are just the bare minimum recommendations, more sophisticated hardening is REQUIRED and HIGHLY EXPECTED.

• Only acces by normal user using SSH keys, so no password login allowed.
• Root should only have access to the console and not to the virtual terminals.

Only the really necessary ports should be exposed to the outside world.

ufw allow <YOUR_NODE_PORT>/tcp
ufw allow <WHERE YOUR SSH SRV IS LISTENING default:22>/tcp

Note: It will allow connection from both IP protocol (IPv4/IPv6).