Skip to content

Instantly share code, notes, and snippets.

@Linw00d93
Forked from jonsoini/elastic-pi.md
Created October 23, 2019 15:41
Show Gist options
  • Save Linw00d93/615fe97929f1237fa86a98ea1247e32c to your computer and use it in GitHub Desktop.
Save Linw00d93/615fe97929f1237fa86a98ea1247e32c to your computer and use it in GitHub Desktop.

Installing elastic beats on Raspberry Pi

At time of writing elastic.co does not provide ARM builds for raspberry. This tutorial describes how to compile e.g. filebeat and run in on Raspberry Pi3.

Prerequisites

You'll need Git, Go (>1.8) and Python with virtualenv.

Go 1.8+ (currently 1.11)

Raspian/Debian stretch comes with Go 1.7. To get something newer you'll need to add the next version buster to the package sources.

Add this line to /etc/apt/sources.list

http://raspbian.raspberrypi.org/raspbian/ buster main contrib non-free rpi

Make sure your /etc/apt/preferences looks like this:

    Package: *
    Pin: release n=stretch
    Pin-Priority: 500

    Package: *
    Pin: release n=buster
    Pin-Priority: 10

Then update the package list:

sudo apt update

And install from buster:

sudo apt install -t buster golang

This should get you Go 1.11 at time of writing.

Python

Python 2.x should come preinstalled with Raspbian but you may be missing pip and virtualenv:

sudo apt-get install python-pip
sudo pip install virtualenv

Memory

Compiling with Go, even using a single CPU core can take a lot of memory. Update /etc/dphys-swapfile to 1024MB instead of the default 100MB swap memory and enable it:

sudo nano /etc/dphys-swapfile
sudo dphys-swapfile setup
sudo dphys-swapfile swapon

Building

Install Gox (Go crosscompiling) from here:

https://github.com/mitchellh/gox

Get the sources

Set the root path for Go package management:

export GOPATH=~/go

Get sources:

go get github.com/elastic/beats

You'll get an error message about valid targets or something, don't worry.

Goto desired beats folder:

cd ~/go/src/github.com/elastic/beats/metricbeat/

Select desired version, should match your elasticsearch backend, in my case 7.1 (note patch versions may not yield a result, so pick '7.1' instead of '7.1.1'):

git checkout 7.1

Set Gox to build for Arm, and build!

gox -osarch="linux/arm"

Build binary

Run the go compiler:

GOPATH=~/go make

This wil output the executable filebeat in the current directory, verify by running:

./filebeat -v -e

where -e will output errors to the console instead of syslog. filebeat will now complain about missing config file filebeat.yml.

Copy filebeat.default.yml and modify as required.

Configuration templates

The fields.yml is required to configure the index. To build run:

make python-env
make fields
mv _meta/fields.generated.yml ./fields.yml

Build the Kibana template:

make kibana
mv _meta/kibana .

Now run filebeat setup:

./filebeat setup -v -e

Add-ons

For some beats plugins the elasticsearch core can be supplied with addons. Assuming you're running the elastic server in a docker image named elasticsearch install addons like this:

sudo docker exec -it elasticsearch bash
cd /opt/elasticsearch/
bin/elasticsearch-plugin install ingest-user-agent
bin/elasticsearch-plugin install ingest-geoip
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment