bonsfi/challenge003.md Secret

## challenge003.md

      
    Raw
  

              challenge003.md
            
          
    There are many guides currently for setting up your node with Prometheus and
Grafana, but for a slightly more lightweight solution for micro-teams there are
other alternatives too. In this guide, we will setup Prometheus node exporter
as well as masknetgoal634's NEAR exporter.
This will allow us to export NEAR metrics from our machine and takes very little
time to setup.
We can monitor these metrics through Netdata.
Setting Up Netdata

Requirements & Setup

First, you will need to install Docker. This is out of the scope of this guide,
once installed, run the relevant containers:
Prometheus Node Exporter

sudo docker run -d \
    -it \
    -p 9100:9100 \
    --name node-exporter \
    --restart always \
    --volume /:/rootfs:ro \
    --volume /proc:/host/proc:ro \
    --volume /sys:/host/sys:ro \
    prom/node-exporter:latest \
        --path.procfs=/host/proc \
        --path.sysfs=/host/sys
NEAR Exporter

Make sure to change <YOUR_POOL_ID> below, you can find this by running
near validators current and copying out your pool ID from the list.
sudo docker run -d \
    -it \
    -p 9333:9333 \
    --name near-exporter \
    --restart always \
    --network=host \
    masknetgoal634/near-prometheus-exporter:latest \
        /dist/main -accountId <YOUR_POOL_ID>
NETDATA

Finally, we can run netdata itself. Note that we must provide netdata with the
host network for it to find the endpoint providing near metrics.

NOTE: Make sure you have an active firewall before running this step or you
will expose netdata to the internet, which is not a great idea.

sudo docker run -d \
    --name=netdata \
    -p 19999:19999 \
    -v /proc:/host/proc:ro \
    -v /sys:/host/sys:ro \
    -v /var/run/docker.sock:/var/run/docker.sock:ro \
    --cap-add SYS_PTRACE \
    --security-opt apparmor=unconfined \
    --network=host \
    netdata/netdata

Check Netdata

Netdata can now be accessed at <SERVER_IP>:19999 (make sure you're configuring
your firewall well) -- below the prometheus exporter you'll find several near
monitoring metrics graphed. No configuration is needed, netdata should find
these automatically.

Setup Pushover and Alerts

If you are happy with monitoring alone you can stop here.
Pushover is a service that allows pushing arbitrary messages to mobile devices,
with various alarm levels and priorities. Netdata has built-in support for
sending alerts via Pushover. Before we get setup, you will need to follow the
following checklist:

 Register and Install Pushover on at least one device.
 Create a Pushover app in the control panel.
 Copy the app token and the user token and have them ready.

Configuring Netdata

We can now configure Pushover alerts in netdata, first we'll need to get a copy
of the configuration file from netdata:
sudo docker cp netdata:/usr/lib/netdata/conf.d/health_alarm_notify.conf .
Inside this configuration file, find the following options and update the <>
values with the keys you copied above:
# ...

PUSHOVER_APP_TOKEN="<APP_TOKEN_FROM_PUSHOVER>"
DEFAULT_RECIPIENT_PUSHOVER="<USER_TOKEN_FROM_PUSHOVER>"

# ...
We're almost there! We can now stop and remove the docker container, and restart
it with the new alarm configuration mounted:
sudo docker run -d \
    --name=netdata \
    -p 19999:19999 \
    -v /proc:/host/proc:ro \
    -v /sys:/host/sys:ro \
    -v /var/run/docker.sock:/var/run/docker.sock:ro \
    -v $PWD/health_alarm_notify.conf:/usr/lib/netdata/conf.d/health_alarm_notify.conf \
    --cap-add SYS_PTRACE \
    --security-opt apparmor=unconfined \
    --network=host \
    netdata/netdata
And we're done! You can now monitor your node, NEAR stats, and get alarms sent
to your device to keep track of your node health.