bwbaugh/logstash-guide.md

## logstash-guide.md

      
    Raw
  

              logstash-guide.md
            
          
    Installing logstash

Server install

Install

Install the Java prerequisite:
sudo aptitude install openjdk-7-jre-headless

Set up the package repository so that we can use aptitude
(docs):
wget -qO - https://packages.elasticsearch.org/GPG-KEY-elasticsearch | sudo apt-key add -
echo "deb http://packages.elasticsearch.org/logstash/1.5/debian stable main" | sudo tee -a /etc/apt/sources.list

Install logstash itself:
sudo aptitude install logstash

Generate SSL certificates

Docs: https://www.digitalocean.com/community/tutorials/how-to-use-logstash-and-kibana-to-centralize-and-visualize-logs-on-ubuntu-14-04
Create the needed directories:
sudo mkdir -p /etc/pki/tls/{certs,private}

Create FQDN (DNS) based certificates (change CN as appropriate):
cd /etc/pki/tls; sudo openssl req -subj '/CN=*.bwbaugh.com/' -x509 -days 3650 -batch -nodes -newkey rsa:4096 -keyout private/logstash-forwarder.key -out certs/logstash-forwarder.crt

Configure

Lumberjack input

Choose a port.

Replace the 5000 example below.
Add the port to the inbound rules for the security group.

Create and edit a configuration file for the logstash-forwarder:
sudo vim /etc/logstash/conf.d/01-lumberjack-input.conf

with the following contents:
input {
  lumberjack {
    port => 5000
    type => "logs"
    ssl_certificate => "/etc/pki/tls/certs/logstash-forwarder.crt"
    ssl_key => "/etc/pki/tls/private/logstash-forwarder.key"
  }
}

Syslog

Create the file:
sudo vim /etc/logstash/conf.d/10-syslog.conf

with the following contents:
filter {
  if [type] == "syslog" {
    grok {
      match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname} %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
      add_field => [ "received_at", "%{@timestamp}" ]
      add_field => [ "received_from", "%{host}" ]
    }
    syslog_pri { }
    date {
      match => [ "syslog_timestamp", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss" ]
    }
  }
}

Debugging output

Create:
sudo vim /etc/logstash/conf.d/30-output-stdout.conf

with:
output {
  stdout { codec => json }
}

S3 output

Create:
sudo vim /etc/logstash/conf.d/30-syslog-output.conf

with:
output {
  if [type] == "syslog" {
    s3 {
      region => "us-west-2"
      bucket => "YOUR_BUCKET_NAME_HERE"
      canned_acl => "private"
  
      codec => "json_lines"
  
      size_file => 10485760
      time_file => 60

      prefix => "syslog/"
    }
  }
}

Start it

Start or restart logstash:
sudo service logstash restart

Logs

Check /var/log/logstash/ for the daemon's output
Client install

Install

Set up the package repository:
wget -O - http://packages.elasticsearch.org/GPG-KEY-elasticsearch | sudo apt-key add -
echo 'deb http://packages.elasticsearch.org/logstashforwarder/debian stable main' | sudo tee /etc/apt/sources.list.d/logstashforwarder.list

Install logstash-forwarder itself:
sudo aptitude update
sudo aptitude install logstash-forwarder

Set up SSL certificates

Copy the Logstash server's SSL certificate:
sudo mkdir -p /etc/pki/tls/certs
# Either copy/paste or use `scp`.
sudo vim /etc/pki/tls/certs/logstash-forwarder.crt

Configure

Edit the configuration file:
sudo vim /etc/logstash-forwarder.conf

Under the network section add the following with the proper host and port:
    "servers": [ "logstash_server_private_address:5000" ],
    "timeout": 15,
    "ssl ca": "/etc/pki/tls/certs/logstash-forwarder.crt"

Under the files section add:
    {
      "paths": [
        "/var/log/syslog",
        "/var/log/auth.log"
       ],
      "fields": { "type": "syslog" }
    }

Start it

Start or restart logstash-forwarder:
sudo service logstash-forwarder restart

Useful tips

The logstash-forwarder keeps track of files it has already sent in:
/var/lib/logstash-forwarder/.logstash-forwarder