This guide will cover installing and configuring netdata to turn your CHIP (or other Linux device) into a monitoring server. I will also review the steps needed to configure a "client" device that sends it's netdata metrics to CHIP. It's actually very easy to do, thanks to the fantastic job the Netdata developers did in making setup and configuration easy. Yes, the sheer number of configuration options is daunting, but getting netdata running on CHIP actually requires no configuration at all.
This guide will cover installing and configuring netdata to turn your CHIP into a monitoring server. Netdata monitors hardware, networking, applications (web servers, db servers, mail servers), as well as many other things, and visualizes the data in real-time on a web dashboard. I will also cover how to configure CHIP as a “server” that receives and makes accessible monitoring data from other Linux devices, or clients. Installing Netdata isn’t difficult, and to just get it running on CHIP requires no configuration.
This tutorial is based off of the official documentation found here: https://github.com/firehol/netdata/wiki/Installation, here: https://github.com/firehol/netdata/wiki/Configuration, and here: https://github.com/firehol/netdata/wiki/Replication-Overview (for setting up the client devices).
There's a few ways you can install Netdata. Since we're installing this on CHIP, only one option is available to us: downloading and compiling the source. Don't worry, it's not hard!
The following will attempt to find and install all dependencies It worked well for me, I needed to install no additional packages. If you don't have curl, do: sudo apt-get install curl first.
curl -Ss 'https://raw.githubusercontent.com/firehol/netdata-demo-site/master/install-required-packages.sh' >/tmp/kickstart.sh && bash /tmp/kickstart.sh -i netdata-all
Note: netdata-all installs everything needed to monitor everything that Netdata can monitor. You want this on the main server, but you may not want this on a client if it's a CHIP. If you install Netdata on another CHIP that isn't a server, specify netdata instead of netdata-all to install Netdata with a more minimal set of metrics.
If install-required-packages.sh fails, manually install the dependencies:
sudo apt-get install zlib1g-dev uuid-dev libmnl-dev gcc make git autoconf autoconf-archive autogen automake pkg-config
Now it's time to install Netdata. First, let's download the source code:
git clone https://github.com/firehol/netdata.git --depth=1
cd netdata
sudo ./netdata-installer.sh
netdata-installer.sh will take about 30 minutes. At some points it may appear to freeze during the compilation process, but it's still running. Eventually you will see some color text and a nice ASCII graphic of a graph. This means Netdata is compiled and installed! If you encounter a problem, post it here, and I'll try to help. If you think it's a genuine bug or limitation, let the developers know: https://github.com/firehol/netdata/issues
If all went smoothly, you can now browse to https://your-chips-ip:19999 in your browser (or https://localhost:19999 if browsing from your CHIP) to see the magic that is Netdata. Seriously, it's impressive. It's nice to look at and has a huge (some might say overwhelming) amount of information. Yes, that's all there is to installing Netdata. You don't have to do anything else. However, if you really want to unlock the power of Netdata, you're going to have to install it on some other hosts/clients.
As mentioned previously, you can install Netdata on other hosts that act as clients. They collect their data and transmit it to the receiver, the receiver being your CHIP. To set this up, repeat steps 1 and 2 on another Linux device. After completing both steps on your client, execute this on the client:
sudo cat /var/lib/netdata/registry/netdata.public.unique.id
The output of that command is your machine identifier. You will need this to register the client with the server. Copy the machine id and open stream.conf on the client:
sudo nano /etc/netdata/stream.conf
Under the section that begins [STREAM] change the value for enabled to yes:
enabled = yes
Insert your machine ID after api key = :
api key = Your-Machine-ID
After destination =, put the IP of your CHIP:
destination = Your-Chip-IP-Address
Save and close stream.conf. restart Netdata:
sudo systemctl restart netdata
Finally, we're going to register the previously obtained client's machine id with CHIP. First, open up stream.conf:
sudo nano /etc/netdata/stream.conf
Scroll down to the section that begins, 3. PER SENDING HOST SETTINGS and paste the following before the section that begins [MACHINE_GUID]:
[CLIENT-MACHINE-ID]
enabled = yes
history = 3600
memory mode = save
health enabled = yes
Finally, insert your client's machine id / API key inside the square brackets [CLIENT-MACHINE-ID]. Save and close stream.conf and then restart netdata.
sudo systemctl start restart
Your client should now be sending data to your CHIP. To check this from either the server or the client:
sudo grep STREAM /var/log/netdata/error.log
On the client, you should see output indicating metrics were sent to your CHIP, and CHIP's log should indicate the data was received. On CHIP, it will look something like this:
2017-09-01 02:03:17: netdata: INFO : Host 'linux-pc' with guid '12345678-b5a6-11e6-8a50-00508db7e9c9' initialized
2017-09-01 02:03:17: netdata: INFO : STREAM linux-pc [receive from [192.168.1.23]:12334]: initializing communication...
2017-09-01 02:03:18: netdata: INFO : STREAM linux-pc [receive from [192.168.1.23]:12334]: receiving metrics...`
- (OPTIONAL) Adding KSM support to decrease Netdata's resource usage
Netdata runs pretty good on CHIP as is. One limitation is that history is set to 3600 seconds, or one hour, meaning Netdata graphs only 1 hour of data at a time. I have not experimented with increasing that on CHIP, in part because higher history values impact system resources. The Netdata developers explain:
So, for each dimension of a chart, netdata will need: 4 bytes for the value * the entries of its history. [...] The default history is 3,600 entries, thus it will need 14.4KB for each chart dimension. If you need 1.000 dimensions, they will occupy just 14.4MB. Of course, 3.600 entries is a very short history, especially if data collection frequency is set to 1 second. You will have just one hour of data. For a day of data and 1.000 dimensions, you will need: 86.400 seconds * 4 bytes * 1.000 dimensions = 345MB of RAM.
Since CHIP has very limited memory, we (probably) can’t set history too high. With history = 3600 set, my CHIP has about 13MB of free memory. An alternative is to enable KSM (Kernel Same Page Merging) in the Linux kernel. The Netdata documentation claims that enabling KSM can decrease memory usage by 40-60%. On CHIP, this requires re-compiling the kernel. I'm not going to cover that here, except to reference this guide: http://www.chip-community.org/index.php/HOW-TO_compile_Chip%27s_Linux_kernel_and_modules_on_Chip_itself
Note, these instructions are not specific to CHIP and will help you setup Netdata on any Linux machine that can run it.