Monitoring Your Elastic Stack, with Beats

Step 1 - Elasticsearch and Kibana.md

Download

wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.3.1.tar.gz
wget https://artifacts.elastic.co/downloads/kibana/kibana-5.3.1-darwin-x86_64.tar.gz
wget https://artifacts.elastic.co/downloads/packs/x-pack/x-pack-5.3.1.zip

Extract

tar zxf elasticsearch-5.3.1.tar.gz
tar zxf kibana-5.3.1-darwin-x86_64.tar.gz

Install X-Pack

Make sure you update the path to the file, it must be absolute.

elasticsearch-5.3.1/bin/elasticsearch-plugin install file:///Users/mark/Workspace/Webinar/x-pack-5.3.1.zip
kibana-5.3.1-darwin-x86_64/bin/kibana-plugin install file:///Users/mark/Workspace/Webinar/x-pack-5.3.1.zip

Start

elasticsearch-5.3.1/bin/elasticsearch
kibana-5.3.1-darwin-x86_64/bin/kibana

Step 2 - Logstash.md

Download

wget https://artifacts.elastic.co/downloads/logstash/logstash-5.3.1.tar.gz

Extract

tar zxf logstash-5.3.1.tar.gz

Install X-Pack

Make sure you update the path to the file, it must be absolute.

logstash-5.3.1/bin/logstash-plugin install file:///Users/mark/Workspace/Webinar/x-pack-5.3.1.zip

Config

Use vim/nano/atom etc to edit the logstash.conf file.

xpack.monitoring.elasticsearch.url: ["http://localhost:9200"] 
xpack.monitoring.elasticsearch.username: "logstash_system" 
xpack.monitoring.elasticsearch.password: "changeme

Start

logstash-5.3.1/bin/logstash -e 'input { stdin{} } filter {} output { elasticsearch { user => "elastic" password => "changeme" } }'

Step 3 - Metricbeat.md

Download

wget https://artifacts.elastic.co/downloads/beats/metricbeat/metricbeat-5.3.1-darwin-x86_64.tar.gz
wget https://artifacts.elastic.co/downloads/beats/beats-dashboards/beats-dashboards-5.3.1.zip

Extract

tar zxf metricbeat-5.3.1-darwin-x86_64.tar.gz

Import dashboards

See https://www.elastic.co/guide/en/x-pack/current/beats.html for more info.

metricbeat-5.3.1-darwin-x86_64/scripts/import_dashboards -user elastic -pass changeme -file beats-dashboards-5.3.1.zip

Start

metricbeat-5.3.1-darwin-x86_64/metricbeat -c metricbeat-5.3.1-darwin-x86_64/metricbeat.yml

Step 4 - Metricbeat and Logstash.md

Update config

Update the output to use Logstash instead of directly to Elasticsearch. Make sure you include auth details!

Use vim/nano/atom etc.

vim metricbeat-5.3.1-darwin-x86_64/metricbeat.yml

Start

logstash-5.3.1/bin/logstash -e 'input { beats { port => 5044 } } filter {} output { elasticsearch { hosts => "localhost:9200" manage_template => false index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}" document_type => "%{[@metadata][type]}" } }'
metricbeat-5.3.1-darwin-x86_64/metricbeat -c metricbeat-5.3.1-darwin-x86_64/metricbeat.yml