sfloess/My ELK Setup.md

## My ELK Setup.md

      
    Raw
  

              My ELK Setup.md
            
          
    ELK

I'm a huge fan of Splunk and wanted similar logging functionality at home.  Therefore, all baremetal/VMs/chroot'd environments send rsyslog's to admin-ap via /etc/rsyslog.d/50-default.conf:
*.* @192.168.168.2:514

rsyslogd on admin-ap

This setup is required so that all clients send their logs to one server who forwards to Logstash server v5.6.16.
Configuration

/etc/rsyslog.d/01-json-template.conf

template(name="json-template" type="list") {
         constant(value="{")
                constant(value="\"@timestamp\":\"") 
                property(name="timereported" dateFormat="rfc3339") 
                constant(value="\",\"@version\":\"1") 
                constant(value="\",\"message\":\"") 
                property(name="msg" format="json") 
                constant(value="\",\"sysloghost\":\"") 
                property(name="hostname") 
                constant(value="\",\"severity\":\"") 
                property(name="syslogseverity-text") 
                constant(value="\",\"facility\":\"") 
                property(name="syslogfacility-text") 
                constant(value="\",\"programname\":\"") 
                property(name="programname") 
                constant(value="\",\"procid\":\"") 
                property(name="procid") 
        constant(value="\"}\n") 
}

/etc/rsyslog.d/60-output.conf

# This line sends all lines to defined IP address at port 10514,
# using the "json-template" format template 
*.* @192.168.168.79:10514;json-template

Elasticsearch on cloud-ap

Had no issues whatsoever running Elasticsearch 7.3.0 in a chroot'd environment.  To install:

wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-7.3.0-amd64.deb
apt install ./elasticsearch-7.3.0-amd64.deb

Configuration

/etc/elasticsearch/elasticsearch.yml

cluster.name: FlossWare
path.logs /var/log/elasticsearch
network.host 0.0.0.0
xpack.ml.enabled: false
bootstrap.system_call_filter: false
Logstash on pi-01

6.8.2/7.3.0 - No Go

Had a terrible time running Logstash versions 6.8.27.3.0 on a Raspberry Pi 3.  In both versions I was met with a NullPointerException loading ffi/ffi (or something like that).
5.6.16 - Works

Installing 5.6.16 worked with no issues.  Instructions:

wget https://artifacts.elastic.co/downloads/logstash/logstash-5.6.16.deb
apt install ./5.6.16.deb

controller

Kibana

Cerebro