Harmony package testing

harmony-package-testing.md

Introduction

This document describes the test plan for Harmony's package release using standard packaging system, RPM and Deb packages.

Standard packaging system has many benefits, like extensive tooling, documentation, portability, and complete design to handle different situation.

Package Content

The RPM/Deb packages will install the following files/binary in your system.

• /usr/sbin/harmony
• /usr/sbin/harmony-setup.sh
• /usr/sbin/harmony-rclone.sh
• /etc/harmony/harmony.conf
• /etc/harmony/rclone.conf
• /etc/systemd/system/harmony.service
• /etc/sysctl.d/99-harmony.conf

The package will create harmony group and harmony user on your system. The harmony process will be run as harmony user. The default blockchain DBs are stored in /home/harmony/harmony_db_? directory. The configuration of harmony process is in /etc/harmony/harmony.conf.

Package Manager

Please take sometime to learn about the rpm/dpkg commands used on Redhat/Debian based distributions. There are many other package managers can be used to manage rpm/deb packages like [Apt]https://en.wikipedia.org/wiki/APT_(software), or [Yum]https://www.redhat.com/sysadmin/how-manage-packages

Setup customized repo

You just need to do the setup of testing harmony repo once on a new host. Noted: those repo are for development/testing purpose only. Official production repo will be different.

RPM Package

RPM is supported on Redhat, Fedora, Amazon Linux distributions.

# do the following once to add the development repo
curl -LsSf http://haochen-harmony-pub.s3.amazonaws.com/pub/yum/harmony-dev.repo | sudo tee -a /etc/yum.repos.d/harmony-dev.repo
sudo rpm --import https://raw.githubusercontent.com/harmony-one/harmony-open/master/harmony-release/harmony-pub.key

Deb Package

Deb is supported on Debian, Ubuntu Linux distributionis.

# do the following once to add the development repo
curl -LsSf https://raw.githubusercontent.com/harmony-one/harmony-open/master/harmony-release/harmony-pub.key | sudo apt-key add
echo "deb http://haochen-harmony-pub.s3.amazonaws.com/pub/repo bionic main" | sudo tee -a /etc/apt/sources.list

Test cases

installation

# debian/ubuntu: latest version
sudo apt-get update
sudo apt-get install harmony

# fedora/amazon linux: latest version
sudo yum install harmony

list/install specific version, ex 2.3.4-37

# debian/ubuntu
# list all versions
apt-cache madison harmony

# install specific version
sudo apt-get install harmony=2.3.4-37

# fedora/amazon linux
# list all versions:
yum --showduplicates list harmony

# install specific version
sudo yum install harmony-2.3.4-37

configure/start

# dpkg-reconfigure harmony (TODO)
sudo systemctl start harmony

uninstall

# debian/ubuntu
sudo apt-get remove harmony

# fedora/amazon linux
sudo yum remove harmony

upgrade

# debian/ubuntu
sudo apt-get update
sudo apt-get upgrade

# fedora/amazon linux
sudo yum update --refresh

reinstall

remove and install

Rclone

install latest rclone

# debian/ubuntu
curl -LO https://downloads.rclone.org/v1.52.3/rclone-v1.52.3-linux-amd64.deb
sudo dpkg -i rclone-v1.52.3-linux-amd64.deb

# fedora/amazon linux
curl -LO https://downloads.rclone.org/v1.52.3/rclone-v1.52.3-linux-amd64.rpm
sudo rpm -ivh rclone-v1.52.3-linux-amd64.rpm

do rclone

# validator runs on shard1
sudo -u harmony harmony-rclone.sh /home/harmony 0
sudo -u harmony harmony-rclone.sh /home/harmony 1

# explorer node
sudo -u harmony harmony-rclone.sh -a /home/harmony 0

Setup explorer (non-validating) node

To setup an explorer node (non-validating) node, please run the harmony-setup.sh at first.

sudo /usr/sbin/harmony-setup.sh -t explorer -s 0

to setup the node as an explorer node w/o blskey setup.

Setup new validator

Please copy your blskey to /home/harmony/.hmy/blskeys directory, and start the node. The default configuration is for validators on mainnet. No need to run harmony-setup.sh script.

Start/stop node

• systemctl start harmony to start node
• systemctl stop harmony to stop node
• systemctl status harmony to check status of node

Change node configuration

The node configuration file is in /etc/harmony/harmony.conf. Please edit the file as you needed.

sudo vim /etc/harmony/harmony.conf

Support

Please leave your comments here or added to the github issue. harmony-one/harmony#3305

Hey Leo,

I tested the deb package

Comments :
the harmony installer doesn't :

• add sudo privilege for the harmony user account
• set a shell for the same user
  that prevented me to su - or sudo while logged in with harmony

there is a typo at : sudo dpk -i rclone-v1.52.3-linux-amd64.deb it should be sudo dpkg ...

I needed to login as harmony user to be able to execute harmony-rclone.sh /home/harmony if not, i had the below issue:

2020/08/25 05:22:11 DEBUG : rclone: Version "v1.52.3" starting with parameters ["rclone" "sync" "-vvv" "hmy:pub.harmony.one/mainnet.min/harmony_db_0" "/home/harmony/archive/harmony_db_0"]
2020/08/25 05:22:11 Failed to load config file "/root/.rclone.conf": open /root/.rclone.conf: permission denied

the harmony-rclone.sh is cloning the 4 shards, would be better to accept a paremeter for the shard we want to download and shard 0

harmony-setup.sh seems to work only with the example to build the explorer node, whem I tried harmony-setup.sh -t validator -s 2, it just gave me back the help

So I had to manually edit /etc/harmony/harmony.conf to match a validator setting.

Note, that the explorer harmony.conf file is setting the is_archival to false, i am used to see so far for an explorer to be true

Thanks soph for the feedback.

• fixed the typo
• will improve harmony-rclone.sh
• harmony-setup.sh shouldn't need -s 2 option as the shard number is determined by blskey. I'll add more info.
• will change the is_archival option for the explorer node.

what happens if youre already on systemd and on a non root user, but youre using a different user than this package suggests (harmony)? will it override everything or will it check if any existing users are there/and possible systemd files?

@mindstyle85

what happens if youre already on systemd and on a non root user, but youre using a different user than this package suggests (harmony)? will it override everything or will it check if any existing users are there/and possible systemd files?

with the sudo -u harmony command above, it should technically not overwrite and install everything however the build has still some bugs

Leo,

so is the harmony user never intended to be used for login ? If yes, there is still one issue with rclone :

2020/08/27 04:24:13 DEBUG : rclone: Version "v1.52.3" starting with parameters ["rclone" "sync" "-vvv" "hmy:pub.harmony.one/mainnet.min/harmony_db_1" "/home/harmony/archive/harmony_db_1"]
2020/08/27 04:24:13 Failed to load config file "/root/.rclone.conf": open /root/.rclone.conf: permission denied

the only way I found to fix that is to login as harmony user and do the rclone, if not the above command still assume the config file is under his the root home directory despite the sudo -u harmony

use sudo -u harmony harmony-rclone.sh /home/harmony 0

I suggest start a new node using the package. For existing node, the migration may not be clean.

that was already a brand new node i was testing.

root@soph-ubuntu-s-1vcpu-2gb-sfo3-01:~# sudo -u harmony harmony-rclone.sh /home/harmony 0
mv: cannot stat '/home/harmony/archive/harmony_db_0': No such file or directory
root@soph-ubuntu-s-1vcpu-2gb-sfo3-01:~# cat /home/harmony/archive/archive-0.log
2020/08/27 04:35:07 DEBUG : rclone: Version "v1.52.3" starting with parameters ["rclone" "sync" "-vvv" "hmy:pub.harmony.one/mainnet.min/harmony_db_0" "/home/harmony/archive/harmony_db_0"]
2020/08/27 04:35:07 Failed to load config file "/root/.rclone.conf": open /root/.rclone.conf: permission denied
root@soph-ubuntu-s-1vcpu-2gb-sfo3-01:~#