Harmony Node Binary CLI

Hidden flags.md

Legacy and hidden flags

This document gives a full list of legacy and hidden flags at harmony CLI interface. There are briefly two types of legacy flags:

1. Flag is only for developer usage, thus is hidden from user to have a simple interface.
2. Flag is a legacy from previous versions and exist for backward compatibility.

1. Hidden flags

1.1 IP and Port

--port

Harmony binary used to have one port settings and all other port is a magic math from this base port:

1. P2P port use exactly this port val
2. HTTP port use val + 500
3. Websocket port use val + 800
4. DNS port use val - 3000

Port is also persisted in each log message and exist in log file name.

--ip

Harmony used to have IP as the public IP of a node only for logging message display, but does not effect the address the service is listening on. In latest implementation, the IP field is defined as the address listen on. 0.0.0.0 for public endpoint.

IP is also persisted in each log message and exist in log file name.

--public_rpc

If set to true, open public HTTP and WS endpoints. Equivalent to --http.ip=0.0.0.0 --ws.ip=0.0.0.0.

--log.ctx.ip & --log.ctx.port

IP and port used for log message and log file name. legacy of --ip and --port fields. Will remove in the future if no service interruption.

1.2 Dev Only (Hidden)

--consensus.min-peers --min_peers

Dev only. Start consensus only after this number of peers are discovered. Default: 6.

--devnet.num-shard & --dn_num_shards / --devnet.shard-size & --dn_shard_size / --devnet.hmy-node-size & --dn_hmy_size

If network is of type devnet, apply these fields in shard schedule.

--revert.to --revert.do-before --revert.beacon / --revert_beacon --do_revert_before --revert_to

Revert the blockchain in given condition.

1.3 No longer available

--block_period

No longer available. Hard coded 5s finality according to ChainConfig.

--delay_commit

No longer available. Default to 0.

--max_bls_keys_per_node

No longer available. Now number of BLS keys per node is hard coded as 10.

1.4 Same name but meaning changed

--pprof

Used to use as --pprof.addr, now use two flags: --pprof and --pprof.addr.

1.5 Legacy with new flags as replacement

--node_type

Legacy, equivalent to --run

--staking

Legacy. Staking mode is now enabled by default. Equivalent to --run.legacy=false

--shard_id

Legacy, equivalent to --run.shard.

--db_dir

Legacy, equivalent to --datadir

--dns_zone

Legacy, equivalent to --dns.zone

--dns_port

Legacy, equivalent to --dns.port

--dns

Legacy. Now dns is enabled by default. Use --dns=false to use legacy syncing.

--network_type

Legacy, equivalent to --network.

--key

Legacy, equivalent to --p2p.keyfile

--blskey_file

Legacy, equivalent to bls.keys.

--blsfolder

Legacy, equivalent to --bls.dir

--blspass

Legacy. Use --bls.pass, --bls.pass.src, --bls.pass.file. Used to have argument --blspass=file:mykey.pass blspass=stdin.

--save-passphrase

Legacy, equivalent to --bls.pass.save.

--aws-config-source

Legacy. Use --bls.kms, --bls.kms.src, --bls.kms.config.

--blacklist

Legacy, equivalent to --txpool.blacklist

--log_folder

Legacy, equivalent to --log.path

--log_max_size

Legacy, equivalent to --log.max-size

--verbosity

Legacy, equivalent to --log.verbosity

2. Full config with hidden fields

Following is the full config file:

Version = "1.0.0"

[BLSKeys]
  KMSConfigFile = ""
  KMSConfigSrcType = "shared"
  KMSEnabled = true
  KeyDir = "./.hmy/blskeys"
  KeyFiles = []
  MaxKeys = 10
  PassEnabled = true
  PassFile = ""
  PassSrcType = "auto"
  SavePassphrase = false

[Consensus]
  MinPeers = 6

[Devnet]
  HmyNodeSize = 10
  NumShards = 2
  ShardSize = 10

[General]
  DataDir = "./"
  IsArchival = false
  NoStaking = false
  NodeType = "validator"
  ShardID = -1

[HTTP]
  Enabled = true
  IP = "127.0.0.1"
  Port = 9500

[Legacy]
  TPBroadcastInvalidTxn = true
  WebHookConfig = "web hook"

[Log]
  FileName = "validator-127.0.0.1-9000.log"
  Folder = "./latest"
  RotateSize = 100
  Verbosity = 3

  [Log.Context]
    IP = "127.0.0.1"
    Port = 9000

[Network]
  BootNodes = ["/ip4/100.26.90.187/tcp/9874/p2p/Qmdfjtk6hPoyrH1zVD9PEH4zfWLo38dP2mDvvKXfh3tnEv","/ip4/54.213.43.194/tcp/9874/p2p/QmZJJx6AdaoEkGLrYG4JeLCKeCKDjnFz2wfHNHxAqFSGA9","/ip4/13.113.101.219/tcp/12019/p2p/QmQayinFSgMMw5cSpDUiD9pQ2WeP6WNmGxpZ6ou3mdVFJX","/ip4/99.81.170.167/tcp/12019/p2p/QmRVbTpEYup8dSaURZfF6ByrMTSKa4UyUzJhSjahFzRqNj"]
  DNSPort = 9000
  DNSZone = "t.hmny.io"
  LegacySyncing = false
  NetworkType = "mainnet"

[P2P]
  KeyFile = "./.hmykey"
  Port = 9000

[Pprof]
  Enabled = false
  ListenAddr = "127.0.0.1:6060"

[Revert]
  RevertBeacon = false
  RevertBefore = 0
  RevertTo = 0

[TxPool]
  BlacklistFile = "./.hmy/blacklist.txt"

[WS]
  Enabled = true
  IP = "127.0.0.1"
  Port = 9800

Introduction.md

Harmony Node Binary CLI

This document is the handbook of node binary options to start a harmony node using harmony binary CLI interface.

Harmony node binary can be started with:

1. CLI flags parsing
2. A config file
3. Flag and config file combined

1. Starting Node with Flags

A full list of active flags as well as examples can be access through running binary with --help option.

./harmony --help

harmony is the Harmony node binary file

Examples usage:

# start a validator node with default bls folder (default bls key files in ./.hmy/blskeys)
    ./harmony

# start a validator node with customized bls key folder
    ./harmony --bls.dir [bls_folder]

# start a validator node with open RPC endpoints and customized ports
    ./harmony --http.ip=0.0.0.0 --http.port=[http_port] --ws.ip=0.0.0.0 --ws.port=[ws_port]

# start an explorer node
    ./harmony --run=explorer --run.archive --run.shard=[shard_id]

# start a harmony internal node on testnet
    ./harmony --run.legacy --network

Usage:
  harmony [flags]
  harmony [command]

Available Commands:
  dumpconfig  dump the config file for harmony binary configurations
  help        Help about any command
  version     print version of the harmony binary

Flags:
      --bls.dir string            directory for BLS keys (default "./.hmy/blskeys")
      --bls.keys strings          a list of BLS key files (separated by ,)
      --bls.kms                   enable BLS key decryption with AWS KMS service (default true)
      --bls.kms.config string     json config file for KMS service (region and credentials)
      --bls.kms.src string        the AWS config source (region and credentials) for KMS service (shared, prompt, file) (default "shared")
      --bls.pass                  enable BLS key decryption with passphrase (default true)
      --bls.pass.file string      the pass file used for BLS decryption. If specified, this pass file will be used for all BLS keys
      --bls.pass.save             after input the BLS passphrase from console, whether to persist the input passphrases in .pass file
      --bls.pass.src string       source for BLS passphrase (auto, file, prompt) (default "auto")
      --bootnodes strings         a list of bootnode multiaddress (delimited by ,)
  -c, --config string             load node config from the config toml file.
      --datadir string            directory of chain database (default "./")
      --dns.port int              port of customized dns node (default 9000)
      --dns.zone string           use customized peers from the zone for state syncing
  -h, --help                      help for harmony
      --http                      enable HTTP / RPC requests (default true)
      --http.ip string            ip address to listen for RPC calls. Use 0.0.0.0 for public endpoint (default "127.0.0.1")
      --http.port int             rpc port to listen for HTTP requests (default 9500)
      --log.dir string            directory path to put rotation logs (default "./latest")
      --log.max-size int          rotation log size in megabytes (default 100)
      --log.name string           log file name (e.g. harmony.log) (default "harmony.log")
  -v, --log.verb int              logging verbosity: 0=silent, 1=error, 2=warn, 3=info, 4=debug, 5=detail (default 3)
  -n, --network string            network to join (mainnet, testnet, pangaea, localnet, partner, stressnet, devnet) (default "mainnet")
      --p2p.keyfile string        the p2p key file of the harmony node (default "./.hmykey")
      --p2p.port int              port to listen for p2p protocols (default 9000)
      --pprof                     enable pprof profiling
      --pprof.addr string         listen address for pprof (default "127.0.0.1:6060")
      --run string                run node type (validator, explorer) (default "validator")
      --run.archive               run node in archive mode
      --run.legacy                whether to run node in legacy mode
      --run.shard int             run node on the given shard ID (-1 automatically configured by BLS keys) (default -1)
      --txpool.blacklist string   file of blacklisted wallet addresses (default "./.hmy/blacklist.txt")
  -V, --version                   display version info
      --ws                        enable websocket endpoint (default true)
      --ws.ip string              ip endpoint for websocket. Use 0.0.0.0 for public endpoint (default "127.0.0.1")
      --ws.port int               port for websocket endpoint (default 9800)

Use "harmony [command] --help" for more information about a command.

2. Starting Node with Config File

All the start options can be persisted and loaded from a single config file. To start a node, the following steps are also available.

1. Dump the default config.
2. Customize the config file.
3. Run harmony node with the config file.

2.1. Dump the default config

./harmony dumpconfig harmony.conf

A file harmony.conf is created and the default node options are set in the file in TOML formatting. Here is an example:

Version = "1.0.0"

[BLSKeys]
  KMSConfigFile = ""
  KMSConfigSrcType = "shared"
  KMSEnabled = true
  KeyDir = "./.hmy/blskeys"
  KeyFiles = []
  MaxKeys = 10
  PassEnabled = true
  PassFile = ""
  PassSrcType = "auto"
  SavePassphrase = false

[General]
  DataDir = "./"
  IsArchival = false
  NoStaking = false
  NodeType = "validator"
  ShardID = -1

[HTTP]
  Enabled = true
  IP = "127.0.0.1"
  Port = 9500

[Log]
  FileName = "harmony.log"
  Folder = "./latest"
  RotateSize = 100
  Verbosity = 3

[Network]
  BootNodes = ["/ip4/100.26.90.187/tcp/9874/p2p/Qmdfjtk6hPoyrH1zVD9PEH4zfWLo38dP2mDvvKXfh3tnEv","/ip4/54.213.43.194/tcp/9874/p2p/QmZJJx6AdaoEkGLrYG4JeLCKeCKDjnFz2wfHNHxAqFSGA9","/ip4/13.113.101.219/tcp/12019/p2p/QmQayinFSgMMw5cSpDUiD9pQ2WeP6WNmGxpZ6ou3mdVFJX","/ip4/99.81.170.167/tcp/12019/p2p/QmRVbTpEYup8dSaURZfF6ByrMTSKa4UyUzJhSjahFzRqNj"]
  DNSPort = 9000
  DNSZone = "t.hmny.io"
  LegacySyncing = false
  NetworkType = "mainnet"

[P2P]
  KeyFile = "./.hmykey"
  Port = 9000

[Pprof]
  Enabled = false
  ListenAddr = "127.0.0.1:6060"

[TxPool]
  BlacklistFile = "./.hmy/blacklist.txt"

[WS]
  Enabled = true
  IP = "127.0.0.1"
  Port = 9800

2.2. Customize the config file

The content of the config file can be modified for custom node start up command.

For example, to open the public HTTP RPCs, change the field IP under [HTTP] tag to "0.0.0.0":

[HTTP]
  Enabled = true
  IP = "0.0.0.0"
  Port = 9500

To run harmony internal nodes under legacy mode instead of staking mode, change the field NoStaking under [General] tag to true:

[General]
  DataDir = "./"
  IsArchival = false
  NoStaking = true
  NodeType = "validator"
  ShardID = -1

2.3. Start the node with config file

Harmony node binary is able to start with options provided by the config file.

./harmony -c harmony.conf

The values stored in config file will be read and parsed to harmony as node start options.

3. Provide both config file and flag

If both config file and flag is provided, the node option stored in config file will be override by the values given in flag.

For example, In config file harmony.conf, HTTP server is enabled, and is open to public:

[HTTP]
  Enabled = true
  IP = "0.0.0.0"
  Port = 9500

And a flag is also provided during the node start command to disable the HTTP server:

./harmony -c harmony.conf --http=false

In this case, the command line flags will override the settings in config file and thus the HTTP server is disabled.

> curl localhost:9500
curl: (7) Failed to connect to localhost port 9500: Connection refused

Conclusion

1. Harmony binary provide flags as the node start options.
2. Custom config file can also be used as a source for start options.
3. If both flag and config file is provided, the value in flags will override the config file settings.

More Explanation.md

More Explanation on Harmony Node Options

This document is some detailed explanation on some of the harmony node start options that might cause confusion.

1. BLS Config

BLS keys are used for validator node to join and participate in Harmony consensus. A bunch of BLS flags are provided for loading BLS keyss:

--bls.dir string             directory for BLS keys (default "./.hmy/blskeys")
--bls.keys strings           a list of BLS key files (separated by ,)
--bls.kms                    enable BLS key decryption with AWS KMS service (default true)
--bls.kms.config string      json config file for KMS service (region and credentials)
--bls.kms.src string         the AWS config source (region and credentials) for KMS service (shared, prompt, file) (default "shared")
--bls.pass                   enable BLS key decryption with passphrase (default true)
--bls.pass.file string       the pass file used for BLS decryption. If specified, this pass file will be used for all BLS keys
--bls.pass.save              after input the BLS passphrase from console, whether to persist the input passphrases in .pass file
--bls.pass.src string        source for BLS passphrase (auto, file, prompt) (default "auto")

1.1 BLS key files

BLS key files can be defined by either of the two flags:

--bls.dir string             directory for BLS keys (default "./.hmy/blskeys")
--bls.keys strings           a list of BLS key files (separated by ,)

• bls.dir defines a directory where all the bls keys are loaded from. By default, directory ./.hmy/blskeys is used.
• bls.keys provide a list of files to being used for consensus.

If both of the options are specified, bls.keys will override bls.dir as the source of BLS flags.

1.2 BLS key decryption

Two decryption algorithm is supported for Harmony BLS keys. Each decryption is bind with a unique extension to instruct the node which decryption to be used for a certain file.

• Passphrase: decrypt the BLS key with a passphrase. The extension of key file is .key

• KMS: AWS Key Management Service. The extension of key file is .bls

1.2.1 Passphrase options

--bls.pass                   enable BLS key decryption with passphrase (default true)
--bls.pass.file string       the pass file used for BLS decryption. If specified, this pass file will be used for all BLS keys
--bls.pass.save              after input the BLS passphrase from console, whether to persist the input passphrases in .pass file
--bls.pass.src string        source for BLS passphrase (auto, file, prompt) (default "auto")

Default behaviour

By default, passphrase is given from source "auto", which means the node will get the passphrase as follows:

1. Try to find the corresponding pass file for the given BLS key. E.g, For BLS key file ./.hmy/blskeys/mybls.key, the passfile is expected to be ./.hmy/blskeys/mybls.pass.
2. The content of the pass file is expected to be the plain text of the passphrase. If the pass file in step 1 is found, node will try to decrypt the BLS key with the passphrase from pass file. If the decryption succeed, the BLS key is loaded to consensus.
3. If the pass file is not found or decryption failed in step 2, prompt the user to enter passphrase through a console password input, and try to decrypt with the provided passphrase.

Extra options

• bls.pass
  • Enable passphrase decryption. Set to false --bls.pass=false if want to disable the passphrase decryption.

• bls.pass.src
  • auto - default option. First try to load passphrase from .pass file. If failed, prompt the user for passphrase.
  • file - use file only and disable prompt. Recommend to use this option if node is running at background (systemd or tmux).
  • prompt - use prompt only and disable load passphrase from file.
• bls.pass.file
  • Use a single passphrase file for all passphrase encrypted keys. If this is set, bls.pass.src will be set to value file.
• bls.pass.save
  • After successfully decrypted a BLS key file with the user prompted passphrase, persist the passphrase to .pass file. Default to false.

1.2.2 KMS options

--bls.kms                    enable BLS key decryption with AWS KMS service (default true)
--bls.kms.config string      json config file for KMS service (region and credentials)
--bls.kms.src string         the AWS config source (region and credentials) for KMS service (shared, prompt, file) (default "shared")

Default behaviour

By default, KMS decryption is enabled and use shared AWS credentials and config settings for BLS key decryption.

Extra options

• --bls.kms.src
  • shared - default option. Use AWS shared credentials and config.
  • prompt - prompt for AWS credentials and region settings.
  • file - provide the AWS config through a json file. If this value is set, a file must be provided for --bls.kms.config
• --bls.kms.file
  • The json config file for AWS credentials and region. If the flag is specified, --bls.kms.src is changed to value file.

2. IP and Ports

There can be multiple IP and ports that the node is listening to. By default, they are:

1. Harmony P2P protocols: listen to Harmony P2P protocol.
2. HTTP RPC: Handle HTTP requests.
3. WS RPC: Handle WS requests.

2.1. P2P

--p2p.port int               port to listen for p2p protocols (default 9000)

Node is listening on p2p.port for harmony protocol which is built on top of LibP2P. Node is always listening to public network traffic, and do not provide customized option for listening IP. User can define customized port through flag --p2p.port which is 9500 by default.

2.2. HTTP

--http                       enable HTTP / RPC requests (default true)
--http.ip string             ip address to listen for RPC calls. Use 0.0.0.0 for public endpoint (default "127.0.0.1")
--http.port int              rpc port to listen for HTTP requests (default 9500)

• By default, the node is only listening on local HTTP RPC requests (127.0.0.1).
• To open public endpoint, run command with --http.ip 0.0.0.0 to handle public HTTP requests.
• To disable HTTP requests, run command with --http=false.

2.3 Websocket

--ws                         enable websocket endpoint (default true)
--ws.ip string               ip endpoint for websocket. Use 0.0.0.0 for public endpoint (default "127.0.0.1")
--ws.port int                port for websocket endpoint (default 9800)

• By default, the node is only listening on local websocket RPC requests (127.0.0.1).
• To open public endpoint, run command with --ws.ip 0.0.0.0 to handle public websocket requests.
• To disable websocket requests, run command with --ws=false.

3. DNS Service

Currently, Harmony is running centralized node synchronization service called as "DNS". This service config is specified by the following flags:

--dns.port int               port of customized dns node (default 9000)
--dns.zone string            use customized peers from the zone for state syncing

DNS zone, shard id and port combined together defines the endpoint for synchronization requests. For example, mainnet DNS zone is t.hmny.io, thus the DNS service provider for shard 1 is s0.t.hmny.io.

The DNS zone already set for different networks. Do not change values on these flags unless necessary.

Centralized DNS service will be replaced by the decentralized version in the future.