Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?

Installing elastic beats on Raspberry Pi

At time of writing elastic.co does not provide ARM builds for raspberry. This tutorial describes how to compile e.g. filebeat and run in on Raspberry Pi3.

Prerequisites

You'll need Go (>1.8) and Pyhton with virtualenv.

Go 1.8

Raspian/Debian stretch comes with Go 1.7. To get 1.8 you'll need to add the next version buster to the package sources. Make sure your /etc/apt/preferences looks like this:

Package: *
Pin: release n=stretch
Pin-Priority: 500

Package: *
Pin: release n=buster
Pin-Priority: 10

Then update the package list:

sudo apt update

And install from buster:

sudo apt install -t buster golang

Python

Python 2.x should come preinstalled with Raspbian but you're likely missing virtualenv:

sudo pip install virtualenv

Memory

Compiling with Go, even using a single CPU core can take a lot of memory. Update /etc/dphys-swapfile to 1024MB instead of the default 100MB swap memory and enable it:

sudo nano /etc/dphys-swapfile
sudo dphys-swapfile setup
sudo dphys-swapfile swapon

Building

Get the sources

Set the root path for Go package management:

export GOPATH=~/go

Get sources:

go get github.com/elastic/beats

Goto desired beats folder:

cd ~/go/src/github.com/elastic/beats/filebeat/

Select desired version, should match your elasticsearch backend:

git checkout 6.0

Build binary

Run the go compiler:

GOPATH=~/go make

This wil output the executable filebeat in the current directory, verify by running:

./filebeat -v -e

where -e will output errors to the console instead of syslog. filebeat will now complain about missing config file filebeat.yml.

Copy filebeat.default.yml and modify as required.

Configuration templates

The fields.yml is required to configure the index. To build run:

make python-env
make fields
mv _meta/fields.generated.yml ./fields.yml

Build the Kibana template:

make kibana
mv _meta/kibana .

Now run filebeat setup:

./filebeat setup -v -e

Add-ons

For some beats plugins the elasticsearch core can be supplied with addons. Assuming you're running the elastic server in a docker image named elasticsearch install addons like this:

sudo docker exec -it elasticsearch bash
cd /opt/elasticsearch/
bin/elasticsearch-plugin install ingest-user-agent
bin/elasticsearch-plugin install ingest-geoip
@VladoPortos

This comment has been minimized.

Copy link

commented Dec 20, 2017

Thanks a ton ! going to give it a try as soon as I have time.

@andig

This comment has been minimized.

Copy link
Owner Author

commented Dec 20, 2017

I've whipped up a small docker that does the compile job and packages everything similar to the distributed official images, see https://github.com/andig/beats4pi

@VladoPortos

This comment has been minimized.

Copy link

commented Jan 1, 2018

Awesome, I'll try it on Asus tinker board tomorrow if it works there :D this monster is hard to manage now https://imgur.com/gHeUok4 hahahaha

@VladoPortos

This comment has been minimized.

Copy link

commented Jan 1, 2018

@andig
I have successfully made filebeat binary that seems to be working ( did not send data to elastisearch server yet) , there is 1.9 golang in buster now. I have added also swap but when I have compiled metricbeat it never go over 1G of RAM ( our of 2GB on Asus Tinker Board ) so that is not needed I think in this SBC

@inqueue

This comment has been minimized.

Copy link

commented Jan 14, 2018

Use make update to generate configuration files and mapping template as opposed to make fields. Otherwise the mapping template is incomplete.

@Ledragon

This comment has been minimized.

Copy link

commented Mar 13, 2018

Thanks man, this is golden!

@mdfranz

This comment has been minimized.

Copy link

commented Nov 23, 2018

Compile worked with go1.10.5 binaries downloaded. Great gist!

@swedishmike

This comment has been minimized.

Copy link

commented Feb 7, 2019

This worked like a charm - many thanks for a great write-up!

@klacc

This comment has been minimized.

Copy link

commented Apr 15, 2019

I am getting stuck here:


root@raspberrypi:~/go/src/github.com/elastic/beats/filebeat# GOPATH=~/go make
Installing mage v1.8.0 from vendor dir.
bash: mage: command not found
../dev-tools/make/mage.mk:9: recipe for target 'mage' failed
make: [mage] Error 127 (ignored)

any ideas?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.